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19  ABSTRACT  (Continue  on  reverse  dne ternary  and  IdbnSRy  by  Mot*  nueiOeti 

Good  measures  of  productivity  are  necessary  components  of  Air  Force  offorts  to  enhance  productivity.  It  Is 
el  so  Inportent  that  different  wtbodel  ogles  to  nwbnnco  productivity  bo  mluntod  In  Air  Force  nnr1nn*nnti.  Use 
techniques  of  ftodhnck,  goal  totting,  and  incantlvts  art  nathodol agios  that  have  shout  good  potential,  and  writ 
such  evaluation.  Once  the  productivity  ■aanurs—nt  systa*  nos  dovolopod,  It  becamt  the  basis  far  providing  formal 
monthly  feedback  to  porsoonal  and  aanagoaant  about  unit  productivity,  tool  netting  and  incentives  uore  added  ta 
feedback.  Noasuras  evaluating  the  offnctlvenoss  #f  the  approach  mart  token  daring  the  dovolopaaot  aad 
ioplaaootatloo  periods,  goth  productivity  and  attitudes  mart  measured.  The  system  use  developed  for  flue  unite 
in  the  maintenance  end  supply  a  roes  ot  an  operational  Air  Force  hose.  After  development,  the  aooooroaoot  ay*  tan 
wts  used  an  the  heals  far  formal  feedback  that  each  unit  received  every  Moth  for  g  Maths,  taut,  gsal  setting 
uns  added  to  the  feedback  for  another  I  months.  Incutaants  end  starrlHn  Jointly  set  productivity  gnla  tar  tat 
following  nenth.  Finally,  incentives  in  tan  form  of  time  off  marc  added  ta  foedbenh  and  goal  netting.  Balts 
could  receive  a  half  or  a  tall  day  off  each  month,  depending  an  (hair  productivity.  The  rauulta  abound  the  tyutan 
to  be  *  very  effective  method  of  productivity  maaanrauant  aad  aahaacamoat.  Its  taplamantatian  una  affective .  unit 
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personnel  were  cooperative  In  developing  end  using  It,  Mi  It  ikwM  fMi  psychometric  characteristics. 
Coopered  to  the  baseline  porlod,  the  fHdkck  pro  do  cod  by  th o  system  rOMltod  lo  M  average  pta  to 
productivity  of  50t  during  feedback,  7»  daring  too  feedback  plot  pool  sotting  Intorvootlon,  and  PH  nbon 
lacontlvos  worn  oddod.  The  positive  offsets  lostod  over  tins,  nod  coatlaood  nfbnr  tbo  dsgsrtoro  of  tbe 
research  team.  Tho  Incuubents  and  super visors  of  tbo  units  evaluated  tbo  systoo  very  positively.  Tbo 
nonagors  of  tbe  units  have  continued  tbe  spates  on  their  sun,  and  have  repeated  that  It  bn  vend  in  ether 
units  at  tho  base.  Tbe  productivity  anasuresent  approach  used  In  this  effort  proved  very  effective  for  Mr 
force  organizations.  Using  tbo  anasuresent  systea  to  provide  feedback  laprevod  productivity  greatly,  leal 
setting  Increased  productivity  beyond  foodbock,  but  due  to  tho  probable  continuing  effects  ef  feedback,  tbe 
goal  sotting  affect  any  not  have  boon  very  large.  Incentives  did  net  add  aeaaloyfully  to  productivity. 
Different  work  attitudes  under  tbe  systea  were  either  as  good  or  better  than  before  tbe  pregrea.  Tbe 
productivity  uoasureaont  systea  has  a  super  of  advantages,  for  aaanpla.  It  can  be  applied  to  any  level  ef 
tbe  organization,  allows  different  units  to  be  contained  Into  one  aoesureaont  systea,  and  aliens  for  direct 
comparison  of  the  productivity  of  different  units.  The  productivity  aeasurcaoet  systea  and  resetting 
feedback  and  goal  setting  systems  show  groat  promise  for  future  Mr  force  use. 
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This  report  describes  a  field  test  of  a  new  approach  to  measuring  organi¬ 
zational  productivity  and  its  use  hi  the  implementation  of  feedback,  goal  setting, 
and  incentive  systems. 

The  productivity  meaeuremeut  system  was  developed  for  five  units  in  the 
maintenance  and  supply  areas  at  an  operational  Air  Force  base.  The  proAictivity 
measures  derived  from  the  system  were  used  as  dm  basis  for  formal  feedback 
reports  given  to  the  units  monthly  for  five  months.  After  this,  goal  setting  was 
added  to  the  feedback  for  five  months.  Incumbents  and  supervisors  of  the  five 
units  jointly  set  goats  each  month.  Finally,  incentives  were  added  to  the  feed¬ 
back  and  goal  setting.  The  incentives  were  time  off  from  work,  and  units  could 
earn  either  a  half  or  a  full  day  off,  depending  on  their  productivity.  _ _ 

The  results  showed  the  approach  to  be  a  very  effective  method  of  produc¬ 
tivity  measurement  and  enhancement.  It  also  lum  several  other  advantages  such 
as  eggrngating  qualitatively  different  units  into  one  productivity  measure,  and  al¬ 
lowing  for  direct  comparison  of  productivity  across  different  units.  Using  the 
feedback  produced  fay  the  system  resulted  in  an  average  gain  in  productivity  of 
90%  over  *»— M—  across  the  five  unite.  When  goal  setting  was  added,  the  mean 
increase  was  79%  over  baseline.  When  incentives  were  added  to  that,  the  mean 
increase  was  716%  over  baseline.  The  positive  effects  lasted  over  time,  and  con¬ 
tinued  after  the  departure  of  the  research  team.  The  units  have  continued  the 
system  oa  their  own,  and  their  wmgiri  have  requested  that  it  be  used  in  other 
unite  at  the  base. 

ft  was  coarhated  that  feedback  increased  productivity  substantially,  goal 
setting  increased  productivity  further,  aad  incentives  did  not  odd  beyond  this. 
The  goal  setting  end  incentive  conchmioas  were  somewhat  tentative  tern  to  the 
preamrs  of  learning  and  ceiling  effects.  Different  work  attitudes  un¬ 
der  the  program  either  improved  or  wore  unchanged.  CoacKmioaa  about  develop¬ 
ing  productivity  systems  in  Air  Force  mvirnummte  were  discussed. 
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L  INTRODUCTION 


The  problem  of  enhancing  productivity  has  been  a  universal  concern  for 
some  time.  Enhancing  productivity  has  implications  for  our  quality  of  life,  our 
economy,  and  our  competitive  position  in  the  world  marketplace  (Alluisi  &  Meigs, 
1983;  American  Productivity  Center,  1981).  In  addition,  individual  organizations 
are  continually  concerned  about  increasing  their  productivity  in  order  to  improve 
their  operational  effectiveness.  This  concern  for  increasing  productivity  is 
shared  by  the  Air  Force,  and  has  led  the  Air  Force  Human  Resources  Laboratory 
(AFHRL)  to  explore  ways  of  enhancing  productivity  through  research  and  devel¬ 
opment  (R&D)  efforts  in  the  productivity  domain. 

The  present  R&D  effort  was  a  large-scale  field  effort  aimed  at  imple¬ 
menting  a  new  approach  to  measuring  productivity,  and  using  that  approach  to  as¬ 
sess  the  impact  of  feedback,  goal  setting,  and  incentives  on  productivity.  This 
technical  paper  is  one  of  three  produced  by  the  effort,  and  describes  the  entire 
project  and  its  results.  A  second  technical  paper  from  the  project,  entitled 
"Organizational , Productivity  Measurement:  The  Development  and  Evaluation  of  an 
Integrated  Approach"  (Pritchard,  Jones,  Roth,  Stuebing,  &  Ekeberg,  1987a),  pre¬ 
sents  a  more  detailed  description  of  the  development  and  evaluation  of  the  pro¬ 
ductivity  measurement  system  used  in  the  effort.  Although  there  is  some  overlap 
between  the  two  papers,  this  one  is  designed  to  present  the  entire  project, 
whereas  the  second  focuses  primarily  on  the  productivity  measurement  system  it¬ 
self.  The  third  paper  resulting  from  the  project  is  quite  different  in  nature.  It 
is  non-technical  and  designed  to  aid  operational  managers  in  designing  and  im¬ 
plementing  feedback,  goal  setting,  and  incentive  systems.  It  is  entitled 
"Manager's  Guide  to  the  Implementation  of  Feedback,  Goal  Setting,  and  Incentive 
Systems"  (Pritchard,  Jones,  Roth,  Stuebing,  &  Ekeberg,  1987b). 

In  the  present  paper,  we  shall  (a)  briefly  review  the  literature  on  feed¬ 
back,  goal  setting,  incentives,  and  productivity  measurement;  (b)  describe  the 
methods  and  procedures  used  in  the  research;  (c)  describe  the  results  of  the  ef¬ 
fort;  (d)  discuss  and  interpret  the  results;  and  (e)  draw  conclusions. 

IL  REVIEW  OF  THE  LITERATURE 


Freftagfc 

The  positive  effect  of  feedback  on  performance  has  become  one  of  the 
moot  accepted  principles  in  psychology  (see  reviews  by  Ammons,  1956;  Annett, 
1969;  Bilodeau  &  Bilodeau,  1961;  Guzzo,  Jette,  &  Katzell,  1965;  Ilgen,  Fisher,  & 
Taylor,  1979;  Nadler,  1979;  Sassenwratb,  1975;  and  Taylor,  Fisher,  &  Ilgen, 
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1964).  Of  particular  interest  to  organizational  researchers  is  the  host  of  studies 
that  have  demonstrated  the  beneficial  effects  of  feedback  on  individual  perfor¬ 
mance  in  organizations  (e.g.  Dockstader,  Nebeker,  ft  Shumate,  1977;  Ilgen, 
Fisher,  &  Taylor,  1979;  Ivancevich  &  McMahon,  1962;  Pritchard,  Bigby,  Beiting, 
Coverdale,  ft  Morgan,  1961).  Although  most  of  the  feedback  research  has  fo¬ 
cused  on  the  impact  of  feedback  on  individual  performance,  feedback  has  also 
been  shown  to  increase  the  productivity  of  groups  (e.g.  Becker,  1978;  Chobbar  & 
Wallin,  1984;  Nadler,  1979). 

Because  this  technical  paper  concerns  productivity  enhancement,  the  review 
of  the  feedback  literature  will  focus  primarily  on  the  type  of  feedback  that  has 
been  or  could  be  used  in  organizations.  Feedback  in  organizations  consists  pri¬ 
marily  of  three  types:  (a)  feedback  to  the  organization  about  organizational  pro¬ 
cesses  and  attitudes  (e.g.,  survey  feedback);  (b)  performance  appraisal  feed¬ 
back;  and  (c)  regular  feedback  about  some  form  of  output,  such  as  units  com¬ 
pleted  or  number  of  safety  violations  per  month.  This  review  will  focus  pri¬ 
marily  on  this  third  type  of  organizational  feedback  while  including  relevant  con¬ 
tributions  from  research  on  the  first  two  types  of  feedback  where  appropriate. 

Aside  from  establishing  that  feedback  works,  feedback  research  has  prin¬ 
cipally  followed  three  lines  of  inquiry.  The  first  involves  investigating  the  dif¬ 
ferent  dimensions  of  feedback  and  how  a  feedback  system  should  be  designed 
along  those  dimensions  so  that  it  can  have  optimal  effect.  The  second  line  of 
research  has  been  to  examine  the  effects  of  moderator  variables  on  these  dimen¬ 
sions.  The  third  area  of  inquiry  has  been  be  to  examine  the  processes  or  medi¬ 
ating  mechanisms  through  which  feedback  affects  performance. 

Research  on  Feedback  Dimensions 

Types  of  feedback  can  be  described  along  a  series  of  dimensions.  Differ¬ 
ent  types  of  feedback  may  have  different  effects,  and  the  effects  of  feedback 
may  be  optimized  by  understanding  these  different  dimensions.  Examples  of  di¬ 
mensions  of  feedback  that  have  been  investigated  are:  source  of  the  feedback 
(Fisher,  1979;  Greller,  1980;  Greller  ft  Herold,  1975),  positive  versus  negative 
feedback  (Fisher,  1979),  immediate  versus  delayed  feedback  (Beeson,  1973; 
Christian,  1972;  Robinson  &  Kulp,  1970;  Sassenwrath  ft  Younge,  1969),  knowledge 
of  right  versus  wrong  responses  (Longstreth,  1970;  Merrill,  1970;  Wike,  1970), 
verbal  versus  non-verbal  feedback  (Lair  ft  Smith,  1970;  Simpson,  1972),  personal 
versus  impersonal  feetfcack  (Pritchard,  Montagno  ft  Moore,  1978;  Weidner,  1968), 
accurate  versus  inaccurate  feecback  (Griswold,  1970),  information  content 
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(Berman,  Fraser,  A  Theious,  1970),  and  frequency  of  feedback  (Chobbar  & 
Wallin,  1964;  Ivancevich,  Donnelly,  &  Lyon,  1970). 

Greller  (1975)  focused  on  the  sources  of  feedback  in  organizations  and 
proposed  a  four-dimension  feedback  source  taxonomy.  The  four  dimensions 
were  consequences  from  agents,  expressions  from  agents,  task  feedback,  and 
internal  comparisons.  Internal  comparisons  arise  from  the  comparisons  one 
makes  between  one’s  self  and  others  on  the  same  job.  Beyond  these  four  dimen¬ 
sions,  Greller  proposed  that  an  evaluative  continuum  exists  along  each  of  these 
four  dimensions.  When  Greller  tested  this  model,  he  found  partial  support,  but 
found  that  positive  and  negative  feedback  were  not  simple  opposites.  Building  on 
this  line  of  research,  Greller  and  Herold  (1977)  reported  on  a  factor  analysis  of 
questionnaire  data.  They  reported  the  following  five  factors:  negative  feedback 
independent  of  source,  positive  feedback  from  superiors,  positive  feedback  from 
non-hierarchical  others,  internal  criteria,  and  work  flow  feedback. 

Pritchard  and  Montagno  (1978)  reviewed  the  literature  and  listed  14  feed¬ 
back  dimensions,  which  can  be  summarized  as  follows: 

1.  Positive  versus  negative  feedback.  Feedback  can  be  given  for 
correct  behavior  (or  high  performance)  and/or  for  incorrect  be¬ 
havior  (or  low  performance).  The  first  is  positive  feedback;  the 
second  is  negative  feedback. 

2.  Timing  of  feedback.  Refers  to  the  delay  between  performance  and 
the  presentation  of  feedback. 

3.  Specificity.  Refers  to  the  molecularity  of  the  behavior  for  which 
feecfcack  is  given. 

4.  Evaluative  -  Ron-Evaluative.  The  feedback  may  be  purely  de¬ 
scriptive  or  it  may  contain  an  evaluative  component. 

5.  Absolute  -  Comparative.  The  feedback  may  be  information  about 
the  recipient's  performance  (absolute),  or  may  also  provide  infor¬ 
mation  about  performance  relative  to  some  set  of  norms  or  to  the 
performance  of  other  recipients. 

6.  External  -  Internal.  Refers  to  the  source  of  the  information.  In¬ 
ternal  feedback  is  based  on  the  person’s  own  experience  with  the 
task.  External  feedback  comes  from  sources  outside  the  person. 

7.  Personal  -  Impersonal.  Concerns  the  level  of  personal  contact 
between  the  source  and  the  recipient. 

8.  Power  of  Source.  Refers  to  the  degree  to  which  the  source  con¬ 
trols  the  recipient’s  rewards. 

9.  Schedule  of  feedback.  The  feedback  may  be  continuous,  or  it 
may  be  given  at  fixed  intervals  or  variable  intervals. 
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10.  Group  -  Individual.  The  feedback  may  be  provided  at  the  indi¬ 
vidual  or  group  level. 

11.  Comprehensiveness.  Refers  to  the  percentage  of  the  recipient's 
role  that  is  covered  by  the  feedback. 

12.  Formal  -  Informal.  Concerns  the  manner  in  which  the  recipient  is 
given  feedback  (e.g.,  written  vs.  verbal  comment). 

13.  Public  -  Private.  The  feedback  may  be  given  to  the  recipients 
alone  or  shared  with  other  members  of  the  organization. 

14.  Accuracy.  Refers  to  the  extent  to  which  the  feedback  reflects  the 
true  performance  of  the  recipient. 

Pritchard  and  Montagno  (1978)  examined  the  absolute-comparative  and  the 
specificity  dimensions  with  simulated  inventory  control  jobs.  Non-specific  feed¬ 
back  was  operationalized  as  the  number  of  processed  orders  with  one  or  more 
errors;  specific  feedback  gave  information  on  the  type  of  errors  made.  Non¬ 
specific  feedback  was  superior  to  specific  feedback  in  terms  of  increasing  per¬ 
formance.  Within  the  non-specific  condition,  comparative  feedback  was  superior 
to  absolute  feedback;  there  was  no  difference  in  absolute  versus  comparative 
feedback  within  the  specific  condition. 

Following  this  same  line  of  research,  Pritchard,  Montagno  and  Moore 
(1978)  examined  the  effects  of  six  feedback  dimensions  in  a  job  simulation  ex¬ 
periment.  Impersonal  feedback  was  found  to  be  superior  to  personal  feedback. 
High  specificity  was  superior  to  low  specificity  feedback.  Performance  was 
equally  improved  by  group  and  individual  feedback.  Public  versus  private  feed¬ 
back  showed  no  differential  effect.  The  authors  concluded  that  the  best  combi¬ 
nation  of  feedback  procedures  in  this  study  was  impersonal,  high  specificity,  in¬ 
dividual  feedback  in  either  a  public  or  private  format.  This  combination  pro¬ 
duced  a  26%  increase  in  quantity  and  a  27%  decrease  in  errors. 

Continuing  this  series  of  feedback  studies,  Pritchard,  Bigby,  Beiting, 
Coverdale,  and  Morgan  (1961)  varied  the  personal-impersonal  and  absolute-com¬ 
parative  dimensions  in  a  field  study.  They  held  constant  the  remaining  dimen¬ 
sions  so  that  the  feedback  was  specific,  individualized,  private,  comprehensive, 
formal,  external,  and  accurate.  The  feedback  did  not  contain  an  evaluative  com¬ 
ponent,  but  simply  reflected  the  performance  levels  of  the  individual  incumbents. 
Results  indicated  that  feedback  significantly  improved  performance  both  in  terms 
of  quantity  and  quality.  The  authors  concluded  that  there  were  no  differences 
between  impersonal  and  personal  feecfcack,  nor  were  there  any  differences  be¬ 
tween  absolute  and  comparative  feedback.  Thus,  they  concluded  that  the  choice 
of  which  type  of  feedback  to  use  should  best  be  determined  by  the  particular 


circumstances  and  preferences  of  the  unit  to  which  the  feedback  would  be  ap¬ 
plied. 


Payne  and  Hauty  (1955)  reported  that  specific  feedback,  which  they 
termed  "directive  feedback,"  increased  motor  performance  over  less  specific 
feedback.  They  also  found  that  global,  comparative  feedback,  which  they  called 
"incitive  feedback,"  was  superior  to  global,  absolute  feedback. 

Dockstader,  Nebeker,  and  Shumate  (1977)  compared  absolute  and  compara¬ 
tive  feedback.  They  found  that  comparative  feedback  was  superior  to  absolute 
feedback  on  quantity  measures  using  keypunch  operators.  They  theorized  that 
comparative  feedback  facilitated  the  setting  of  implicit  goals  by  the  operators. 

In  examining  the  effects  of  feedback  frequency,  Chobbar  and  Wallin 
(1984)  varied  the  frequency  of  feedback  about  safety  violations  in  a  manufactur¬ 
ing  plant.  They  found  that  frequent  feedback  (once  a  week)  was  no  more  ef¬ 
fective  in  increasing  safety  performance  than  was  less  frequent  feedback  (once 
every  two  weeks).  Examination  of  the  safety  performance  data  indicates  there 
may  have  been  a  ceiling  effect  operating  so  that  there  was  no  opportunity  for  the 
more  frequent  feedback  to  show  a  difference.  Both  feedback  conditions  signifi¬ 
cantly  improved  safety  performance  over  baseline  levels. 

Greller  and  Herold  (1975)  reported  that  the  more  proximal  the  source  of 
the  feedback,  the  greater  the  utilization  of  the  feedback.  As  one  indication  of 
this,  an  individual’s  own  feelings  and  feedback  from  the  task  were  reported  as 
the  most  commonly  used  sources  of  feedback. 

As  a  whole,  these  studies  do  not  identify  the  characteristics  of  the  optimal 
feedback  system.  They  do  show  that  a  variety  of  feedback  approaches  work. 
This  overall  finding  is  quantified  by  the  meta-analysis  of  Guzzo,  Jette,  and 
Katzell  (1985).  Their  results  indicate  that  there  is  a  significant  effect  on 
performance  due  to  feedback  interventions  across  a  variety  of  situations.  The 
authors  determined  that  the  mean  effect  size  for  feedback  interventions 
(including  performance  appraisal)  is  .5  standard  deviations. 

Moderators  of  the  Feedback-Performance  Relationship 

Moderators  of  the  feedback-performance  relationship  are  used  to  explain 
why  a  feedback  effect  may  exist,  or  why  it  is  stronger  in  one  condition  than  it 
is  in  another.  One  of  these  moderator  variables  is  the  initial  level  of  perfor¬ 
mance  or  motivation  of  the  recipients.  Berkowitz,  Levy,  and  Harvey  (1957) 
found  that  initial  motivation  of  the  recipients  affects  the  task-oriented  behavior 
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following  feedback.  Initial  high  motivation  individuals  exhibited  more  task  ori¬ 
ented  behavior  than  did  low  motivation  individuals  after  negative  feedback.  After 
positive  feedback,  initial  low  motivation  individuals  became  more  task-oriented. 

Ilgen,  Fisher,  and  Taylor  (1979)  reviewed  the  feedback  literature  and  de¬ 
veloped  a  model  of  the  feedback  process.  Their  model  allows  for  individual 
differences  to  impact  the  feedback  process  at  four  points:  perception  of  the 
feedback,  acceptance  of  the  feedback,  desire  to  respond  to  the  feedback,  and  in¬ 
tended  response.  They  concluded  that  personality  variables  create  a  frame-of- 
reference  that  causes  the  feedback  to  be  interpreted  so  that  it  is  consistent  with 
the  recipient's  self-image.  Thus,  persons  with  high  self-esteem  rely  less  on 
feedback  from  the  job  environment  and  more  on  their  own  self-perceptions  to 
guide  their  work  behavior  than  do  people  with  low  self-esteem  (Weiss,  1977). 
In  a  related  fashion,  subjects  with  an  internal  locus  of  control  perform  better 
when  the  feedback  originates  directly  from  the  task  itself,  whereas  those  with 
an  external  locus  of  control  perform  better  when  a  person  (e.g.,  an  experimenter 
or  supervisor)  provides  the  feedback  (Baron,  Cowan,  &  Ganz,  1974;  Baron, 
Cowan,  Ganz,  &  McDonald,  1974;  Baron  &  Ganz,  1972). 

Feedback  Mechanisms  and  Processes 


Research  on  feedback  has  also  explored  how  feedback  works.  In  their 
review  of  the  feedback  literature,  Ilgen,  Fisher  and  Taylor  (1979)  commented  on 
the  large  number  of  studies  relating  dimensions  of  feedback  to  responses,  with 
little  attention  to  the  mediating  psychological  processes.  Locke  and  Bryan  (1969) 
noted  the  same  gap  ten  years  earlier:  "the  question  of  how  KR  [knowledge  of  re¬ 
sults]  facilitates  performance  has  not  yet  been  answered"  (p.  89). 

Several  researchers  have  offered  ideas  concerning  how  feedback  influ¬ 
ences  performance.  Ammons  (1956)  suggested  three  mediating  effects  of  feed¬ 
back  which  have  been  elaborated  by  others  (e.g.  Becker,  1978;  Chapanis,  1964): 
(a)  Feedback  informs  the  recipients  about  their  performance  in  terms  of  extent 
and  type  of  performance,  (b)  feedback  rewards  or  punishes  the  recipient  for  ac¬ 
ceptable  or  unacceptable  performance,  and  (c)  feedback  may  motivate  the  recipi¬ 
ent  through  intrinsic  means. 

Several  researchers  have  examined  the  motivating  effect  of  feedback. 
Gibbs  and  Brown  (1955)  reported  that  subjects  who  received  absolute  feedback 
about  the  number  of  units  finished  increased  performance.  The  explanation  was 
that  the  information  provided  led  the  subjects  to  compete  against  themselves. 
Chapanis  (1964),  however,  found  he  could  not  replicate  these  results  for  a  simi¬ 
lar  task. 
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Locke  and  his  associates  have  argued  that  the  motivational  effect  attributed 
to  feedback  is  actually  due  to  goal  setting  (Locke,  Cartledge,  &  Koeppel,  1968). 
According  to  this  view,  feedback  will  have  a  facilitating  motivational  effect  only 
when  it  leads  to  the  setting  of  a  difficult  performance  goal.  This  implies  that 
feedback  is  a  necessary  but  not  sufficient  condition  for  improved  performance. 
There  is  some  supporting  research  for  this  proposition  (Becker,  1978;  Brez, 
1977;  Locke  ft  Bryan,  1968;  Locke,  Shaw,  Saari,  ft  Latham,  1961;  Shaw,  Locke, 
ft  Bobko.  1961). 

It  should  be  noted,  however,  that  Locke  views  conscious  intentions  as 
goals.  That  is,  if  an  individual  decides  that  he/she  will  attempt  to  produce  100 
units,  this  is  goal  setting.  Locke's  position  is  that  this  goal  setting  must  occur 
for  feedback  to  have  a  positive  effect  on  performance.  This  is  quite  different 
than  saying  there  must  be  a  formal  goal  setting  program  for  feedback  to  affect 
performance.  It  is  very  clear  that  feedback  alone  can  have  powerful  effects  on 
performance.  Thus,  the  issue  that  the  above  literature  addresses  is  whether  a 
person  must  have  a  conscious  intention  of  performing  at  a  given  level  in  order 
for  feedback  to  work. 

Following  the  logic  that  feedback  should  lead  to  goal  setting,  Bigby  (1961) 
and  Morgan  (1964)  surveyed  participants  in  a  field  experiment  where  feedback 
resulted  in  improved  performance  (Pritchard,  Bigby,  Beiting,  Coverdale,  &  Mor¬ 
gan,  1961).  Bigby  (1961)  reported  non-significant  decreases  in  personal  goal 
setting  from  baseline  to  feedback.  Morgan  (1964),  using  a  regression  analysis, 
found  that  personal  goal  setting  predicted  only  one  of  four  performance  mea¬ 
sures. 


Pritchard  and  Montagno  (1978)  proposed  four  mediating  mechanisms  to  ex¬ 
plain  the  effects  of  feedback  on  performance:  (a)  improved  role  clarity,  (b)  ex¬ 
pectancy-instrumentality-valence  effects,  (c)  implicit  goal  setting,  and  (d)  direct 
effects  on  intrinsic  motivation.  The  authors  tested  all  of  these  mechanisms  ex¬ 
cept  implicit  goal  setting,  but  found  none  of  the  hypothesized  changes  even  though 
performance  did  increase  as  a  result  of  feedback. 

The  Ilgen,  Fisher,  and  Taylor  (1979)  model  emphasized  characteristics  of 
the  feedback  and  the  recipient  which  influence  several  aspects  of  the  feedback 
situation.  These  include  the  perception  and  acceptance  of  the  feedback  by  the 
recipient;  the  desire  of  the  recipient  to  respond  to  the  feedback;  die  intended  re¬ 
sponse  of  the  recipient;  and  finally,  die  eventual  response  of  the  recipient.  Re¬ 
search  has  supported  portions  of  this  model  (e.g.,  Vandaveer,  1961).  However, 
since  the  model  is  intended  primarily  to  explain  the  performance  appraisal 


feedback  process,  its  contribution  is  limited  in  sspleiein  feedback  of  the  type 
with  which  we  are  concerned  here. 

Nadler  (1979)  prapoeed  a  preliminary  model  of  the  effects  of  feedbeck  on 
group  task  behavior.  Building  on  propositions  by  Payne  and  Hauty  (1995)  and 
Vroom  (1964),  and  a  model  developed  by  Hackman  and  Morris  (1975),  Nadler 
proposed  a  model  of  group  performance  as  it  is  affected  by  feedback.  Nadter 
reviewed  34  studies,  moat  of  which  indicated  that  feedback  can  improve  group 
functioning  through  a  cueing  function,  a  motivational  function,  or  a  combination  of 
the  two. 

Nadler  proposed  that  the  variables  of  aggregation  of  the  feedback  (group 
or  individual),  focus  of  the  feedback  (task  or  process),  and  the  evaluative  con¬ 
tent  of  the  feedback  (positive  or  negative)  exert  their  effects  through  the  cueing 
or  motivating  functions  but  that  the  effects  of  these  variables  are  contingent  on  a 
number  of  factors.  He  suggested  that  the  effect  of  level  of  feedback  aggrega¬ 
tion  on  groups  is  contingent  on:  (a)  the  desired  impnct  of  the  feedbeck  (i.e.,  af¬ 
fective,  cognitive,  or  behavioral),  (b)  the  nature  of  the  group  task,  and  (c)  the 
characteristics  of  the  group  members.  He  found  that  group-level  feedback  en¬ 
hanced  group  attitudes  and  task  oaotivation,  but  that  individual  feedback  woe  more 
effective  in  enhancing  individual  performance.  For  the  second  factor,  he  found 
that  group  feedback  was  more  effective  when  interdependent  tasks  were  per¬ 
formed  by  the  group  end  that  individual  feedback  was  nwre  effective  where 
group  performance  was  simply  the  sum  of  individual  performances.  For  the 
third  factor,  characteristics  of  group  members,  he  found  that  affiliation-oriented 
individuals  were  more  responsive  to  group-level  feedback  than  were  non-affilia¬ 
tion  individuals. 

Nadler  also  examined  the  effects  of  the  evaluative  content  of  feedback  an 
group  performance.  He  concluded  that  evaluative  feedback  does  lead  to  "changes 
in  member  motivation,  Croatian  of  attributes  (including  defensive  coping)  and 
changes  in  patterns  of  group  interaction  or  structure."  Negative  group-level 
feedback  is  said  to  have  an  unclear  pattern  of  effects  on  group  performance, 
since  it  evokes  coping  behaviors  (Zander  A  Wulff,  19(6)  and  external  attributions 
to  the  causes  of  the  group's  poor  performance  (Schl anker,  Sored,  A  McCarthy, 
1976)  and  may  also  result  in  the  lowering  of  group  aapirotiona  (Dustin,  1966). 

Taylor,  Fisher,  and  Qgea  (1984)  reviewed  research  on  individuals'  reac¬ 
tions  to  feedback  taking  a  control  theory  o^rnorl  to  explain  how  feedback  influ¬ 
ences  performance.  They  discussed  the  iooue  of  discrepancies  between  employ¬ 
ees’  performance  standards  and  the  orgauiauAan’s  performance  standards  and  ar¬ 
gued  that  if  feedback  is  to  hove  8m  impact  intended  by  a  source,  8mre  must  be 
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high  congruence  between  behavioral  towdarda  and  pirfnnaanra  dhaenaioao  in¬ 
cluded  ia  the  faadback. 

They  wwtad  four  cauaat  of  differences  between  these  sals  of  stan¬ 
dard*.  First,  faadback  recipients  may  lack  ataadards  to  guide  their  behavior  on 
the  job.  Second,  recipients*  standards  far  behavior  auqr  be  qnlitativdy  different 
from  thoae  of  the  organisation.  Third,  recipients  tuj  have  a  different  priority 
system  for  standards  than  the  organisation  doss.  Fourth,  there  may  be  quantita¬ 
tive  differences  in  the  level  of  standards  held  by  the  the  source  and  the  recipi¬ 
ent. 

These  differences  in  standards  represent  a  major  issue  to  be  addressed  in 
developing  a  feedback  system.  This  is  a  particularly  sensitive  issue  where  the 
feedback  system  contains  standards  or  an  evaluative  dimension.  A  system  that 
does  not  reach  come  resolution  of  conflicts  between  these  standards  may  not  be 
very  effective. 

Taylor  at  al.  (1994)  discussed  the  effects  of  feedback  on  goal  levels. 
They  cited  evidence  that  goals  change  ia  response  to  feedback  ia  the  following 
ways:  (a)  Feedback  produces  variance  ia  participants'  goals,  (b)  Long-term 
negative  feedback  may  lead  to  redhead  goals;  however,  the  level  of  sub-goals 
may  be  increased  ia  response  to  negative  feedback,  (c)  Positive  feedback  may 
influence  standards  by  increasing  them,  decreasing  them,  causing  them  to  remain 
constant,  or  changing  their  nature.  Positive  feedback  may  increase  performance 
when  there  is  no  obvious  payoff  associated  with  surpassing  standards  as  opposed 
to  merely  meeting  them.  Standards  may  also  be  increased  following  positive 
feedback  (Campion  ft  Lord,  1912).  Goals  may  also  remain  stable  after  positive 
feedback  (a  type  of  "keep  up  the  good  work"  attitude).  The  nature  of  standards 
may  also  change  through  the  alteration  of  strategies. 

Taylor  et  al.  also  looked  at  the  potential  behavioral  responses  to  feedback. 
Reminiscent  of  the  cueing  function  and  motivating  function  of  Payne  and  Hauty 
(1955)  and  Nader  (1979),  they  looked  at  ths  effects  of  feedback  on  the  direction 
of  behavior.  They  concluded  that  negative  faadback  seems  to  initiate  a  search 
for  alternative  work  strategies,  la  addition,  they  n^gesta d  that  nagative  feed¬ 
back  may  decranan  sffort  whaa  the  wpartaacias  are  low.  Ths  authors  also  «x- 
aminad  potmtial  rsaponass  against  the  feedback  system.  Even  whaa  feedback  is 
perceived  an  fair,  it  mop  provoke  a  release  of  "psychological  reactance,"  which 
is  a  daairs  to  reassert  cure!  whaa  ana's  f readme  of  choice  it  threatened 
(Brains,  1W;  Brehm  A  Brahm,  19H).  This  concept  of  reactance  may  help  to 
esplaia  wig  mdleterel,  critical,  or  directive  performance  appraisals  and  faadback 
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systems  art  leas  ef  fectivs  than  amre  participative  appraisal  and  feedback  sys¬ 
tems  (e4.,  Bassett  A  Meyer,  IKS;  Kay,  Meyer,  I  Preach,  1965). 


The  benefits  of  rombinisg  foal  settiag  with  feedback  to  aahanra  perfor¬ 
mance  or  prodactivity  have  been  well  established  in  a  wide  variety  of  situations 
(see  reviews  by  Latham  8  Yukl,  1975;  Locke,  1968;  Locke,  Shaw,  Seari,  8 
Latham,  1981;  Steers  8  Porter,  1974;  and  Tubbs,  1986).  Goal  settiag  has  been 
effectively  need  with  a  variety  of  jobs  such  as  production  workers  (Dnchler  8 
Mobley,  1973);  telephone  operators  (Burke  8  Wilcox,  1969);  looisg  crews 
(Latham  8  Yukl,  1975);  Navy  industrial  workers  (Crawford,  White,  8  Magnus- 
son,  1983);  clerical  workers  and  keypunch  operators  (Dockstader,  Nebeker,  8 
Shumate,  1977;  Pritchard,  Bigby,  Baiting,  Coverdale,  8  Morgan,  1981);  truck 
drivers  (Latham  8  Baldes,  1975)  and  engineers  (Ivaocevich  8  McMahon,  1982). 
In  a  meta-analysis  of  psychologically  based  interventions  (Guzso,  Jette,  8  Katzell, 
1965),  goal  setting  was  found  to  have  an  average  effect  size  of  .8  standard  devi¬ 
ations. 


Goal  setting  research  has  primarily  followed  die  same  avenues  that  have 
been  explored  in  the  reeearch  on  feedback:  the  dimensions  of  the  goal  setting  ef¬ 
fect,  possible  moderators,  and  die  process  or  mechanisms  whereby  the  effect  oc¬ 
curs. 

Diqaenfipw  gf  Qggl  Siting 

As  mentioned,  research  has  explored  the  dimensions  of  goal  setting  to  de¬ 
termine  how  goals  should  be  set  for  maximum  effectiveness.  These  dimensions 
are:  goal  difficulty,  goal  specificity,  and  participation  in  the  setting  of  the  goal. 
Research  results  convincingly  demonstrate  that  challenging  and  specific  goals  re¬ 
sult  in  better  performance  than  do  easy  goals,  no  goals,  or  vague  "do  your  best" 
goals  (Locke,  Shaw,  Saari,  8  Latham,  1981;  Tubbs,  1986).  Additionally,  there 
appears  to  be  a  positive,  linear  relationship  between  goal  difficulty  and  perfor¬ 
mance,  provided  8m  goal  has  bean  accepted  (Bret  8  Ztdon,  1984).  Locke,  et  al 
(1981)  ponclmlad  that  the  necessity  of  perticipetioe  in  the  setting  of  the  goal  is 
unclear.  They  reviewed  a  series  of  studies  that  demonstrated  that  when  the 
level  of  gaal  difficulty  is  the  same  for  perticipetively  and  aon-participatively  set 
goals,  there  is  an  difference  in  performers.  Tubbs,  however,  in  Ms  meta-anal¬ 
ysis  fesnd  a  week  but  pasMee  effect  sice  for  participation.  Purtherssore, 
Locke  et  al.  (19M)  auggeeled  that  partiripotinn  stay  have  aSbtie  effects  on  such 
outoames  as  mperviaory  agpert,  as  wall  aa  improved  understanding  of  goals  and 
how  to  rami  them.  In  addition,  it  wee  found  that  perticipetively  set  goals  tend 
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set  higher  thaa  ana  particlpallvsly  Mt  goals  (Latham,  Mitchell,  ft  Doesett, 
ad  8m  tad  to  laid  to  higher  performance. 


Swtral  moderator  wriabla  have  boat  studied  for  Mr  effects  on  the 
goal  attfafpafoniari  relationship.  The  following  oadtrUan  hove  received 
empirical  support  (TUbbe,  1986): 

1.  Laboratory  versue  field  tettiaf.  Goal  aattiag  demonstrates  a 
stronger  effect  ia  the  laboratory  thaa  ia  actual  orgaaUmtiaas. 

2.  Freaeuce  of  aud  formality  of  feedback.  Specific  poals  and  pertici- 
patively  sat  pools  an  aon  affective  whaa  feedback  is  present  aad 
is  formal  rather  thaa  informal. 

3.  Quantity  pool  versus  time  pool.  Goal  setting  has  a  st reaper  effect 
when  the  poal  is  in  terms  of  quantity  rather  than  time. 

Accordiap  to  Locke's  (1968)  theory,  a  poal  is  a  mental  event  that  is  sim¬ 
ply  what  the  individual  nonariouafy  intends  to  accoaapliah.  The  theory  assumes 
that  roaariona  intentions  regulate  actions,  aad  furthermon  that  poals  ere  the  im¬ 
mediate  rapulaton  of  action.  Because  poals  an  seen  as  the  immediate  repula- 
ton  of  action,  than  an  no  cognitive  aiadlatiap  weehan isms  proponed  by  Locke 
and  associates  to  enpiaia  how  pool  sattiap  works.  Instead,  dm  mediating  mecha¬ 
nisms  an  seen  as  actions  that  occur  between  the  establishment  of  the  poal  and 
the  final  targeted  performance  level.  These  mechanisms  an  indicated  below. 

1.  Direction  (of  behavior).  Whan  poals  an  established,  die  poal-rele- 
vant  areas  of  performance  are  attended  to  more  than  are  non-goal- 
rel event  areas  of  performance. 

2.  Effort  Goals  result  hi  higher  perform  sac  a  because  people  work 
harder  than  if  then  an  no  goals,  and  they  work  even  harder  when 
the  goals  an  harder. 

3.  Persistants.  This  mechanism  is  simply  a  combination  of  direction 
aad  effort  over  time. 
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4.  Strategy  Development.  People  with  teals  art  likely  to  develop  ac¬ 
tio*  plans  or  to  Uaprova  st ratafias  for  achieving  tkair  foals. 

b  their  review,  Locke,  at  01.  (1981)  dtad  evidmca  supporting  these  four 
mechanisms.  Support  appears  weakest  for  the  Effort  aad  the  Persistence  mech¬ 
anisms,  while  the  Direction  serf  the  Stratify  Davslopwmt  mochaniama  have  been 
more  thoroughly  roaaerchad. 

Naylor  and  Ufa*  (1904)  reviewed  the  mechanisms  proposed  by  Locke  at  al. 
(1981)  sod  concluded  that  the  mechanisms  were  quite  complex  aad  required 
somewhat  ambiguous  sub-const ructs.  Therefore,  they  propoesd  mechanisms  for 
foal  setting  that  were  derived  directly  from  the  organi rational  theory  propoesd  by 
Naylor,  Pritchard,  and  Qgan  (1980),  and  showed  how  different  types  of  goal  set¬ 
ting  would  be  explained  by  the  theory.  Although  testing  of  soaae  of  their  propo¬ 
sitions  may  be  difficult,  Naylor  and  Ufa*  make  a  persuasive  argument  for  em¬ 
ploying  these  very  precise  concepts  in  explaining  how  goal  setting  works. 

Incentives 

The  use  of  incentives  to  alter  motivation  and  behavior  has  been  researched 
for  over  half  s  century  (Hull,  1943;  Lewis,  1998;  Skinner,  1938;  Tkoradike. 
1911;  Tollman,  1932).  Incentives  have  been  used  is  almost  every  conceivable  situ¬ 
ation  where  a  behavior  is  to  be  anb eared.  Reviews  of  incentive  motivation  theo¬ 
ries  may  be  found  ia  Campbell  and  Pritchard  (1976);  DaLeo  (1972);  Guzsd 
(1979);  and  Lawler  (1971).  Although  moat  formal  reward  systems  in  organina- 
tians  employ  financial  bcantivea,  motivation  thaorisa  hold  that  behavior  can  be 
motivated  by  e  variety  of  rewards  inrhahag  pay,  praiaa,  promotion,  recognition, 
time-off,  social  freedom,  eelf -development,  alleviation  of  boradom,  sense  of 
accompUahmanf,  etc. 


Although  dm  effectiveness  of  incentives  has  been  well  established  in  the 
psychological  end  management  literature,  only  26ft  of  U.S.  workers  are  covered 
by  formal  incentive  plant  aimed  at  mhaacing  productivity  (Fein,  1976).  Incentive 
programs  era  more  prevalent  in  manufacturing  firms  (99ft)  than  in  non-meau- 
facturiag  firms  (6ft)  (Rice,  1977).  Federal  aactor  nrgaaiierinaa,  which  have 
lata  freedom  ia  allocating  funds  for  programs  such  an  incentives,  reportedly 
spent  only  .1ft  of  the  Federal  payroll  for  caab  bcaHvo  awards  b  fiscal  year 
1978.  Only  tbs  small  amount  wan  spent,  even  though  the  savings  to  dm  Govera- 
ment  wart  aadmatad  to  avaragn  811  for  every  H  spent  on  incentive  awards 
(Specter  A  Hayes,  1979). 
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Tagrlor  (1947)  *u  thi  first  to  writ*  about  the  use  of  incentives  in  orga¬ 
nisations.  Ho  ugud  that  people  will  work  harder  when  their  rewards  and  pun- 
ishmaats  are  directly  tied  to  their  perform  ante.  In  addition  to  the  requirement 
of  a  perceived  Hah  between  incentives  and  performance,  the  rewards  must  be 
perceived  aa  worthwhile  and  valuable  if  they  are  to  be  effective  incentives 
(Lawler,  1973).  Also,  the  performance  goals  for  achieving  the  incentives  must 
be  perceived  aa  attainable  if  the  incentives  are  to  have  the  desired  effect.  Much 
theoretical  and  satirical  work  on  expectancy  theory  has  refined  these  ideas 
(Campbell  *  Pritchard,  19716;  Lawler.  1973;  Mitchell.  1974). 

The  batter  the  an^loyee  understands  the  relationship  between  delivery  of 
incentives  and  performance,  the  more  effective  the  incentive  system  will  be 
(Opeahl  A  Dometts,  1966).  A  mass  of  evidence  supports  this  proposition  (Arvey, 
1972;  Atkbmon.  1958;  Atkinson  A  Raitmaa,  1956;  DeLeo,  1972;  Evans,  1970;  Gal¬ 
braith  A  Ownmiags,  1967;  Gavin,  1970;  Green,  1969;  Hackman  A  Porter,  1968; 
Jorganaoa.  Dmnstte,  A  Pritchard,  1973;  Lawlar  A  Porter,  1967;  Porter  &  Lawler, 
1968;  Pritchard  A  Sanders,  1973).  For  example,  Georgopoulos,  Mahoney,  and 
Jonas  (1957)  rapnrtad  that  workars  who  perceived  higher  personal  productivity  as 
a  amana  to  incraaaad  earnings  wars  more  productive  than  workers  who  did  not 
perceive  such  a  ntattamhip. 

The  clarity  of  this  relationship  may  be  diluted  in  various  situations. 
Campbell  (1971)  reported  a  negative  relationship  between  group  size  and  clarity 
of  the  relationship  between  pay  and  productivity.  Lawler  (1964)  reported  that,  on 
the  average,  managers  saw  no  relationship  between  their  performance  and  their 
pay,  aMheugh  highly  awdvntad  managers  did  sat  such  a  relationship. 

Ths  hapnrtanri  of  the  parcsptioos  of  rewards  in  determining  their  effect 
on  nwtivnHan  can  ha  sew  in  the  oonunon  example  of  the  supervisor  giving  praise 
In  the  aUbordhmte.  If  the  subordinate  perceives  the  praise  as  condescending,  in¬ 
sincere,  or  a  tactic  of  ingratiation,  the  praise  may  have  the  opposite  of  the  ef¬ 
fect  iatandsd. 


Wood,  Hakoi,  DalGaiso  and  KUaaoaki  (1975)  reported  on  the  attractiveness 
sad  faaaftiUty  of  wing  social  incentives  hi  Air  Pores  technical  training.  Their 
factor  analytic  revealed  Wat  ranupdtion  was  ths  moat  attractive  dimension,  fol¬ 
lowed  by  personal  freedom,  self  development,  social  behaviors  (such  as  social 
ialaranfhm,  baing  esasornad  for  htlpiag  8m  instructor,  ate.),  and  information 
feedback  (aush  an  Mwmattos  aa  adUtary  aaoigsaMsts,  or  feedback  on  strengths 
and  weaknesses).  Ths  more  attractive  inrndivaa  ware  those  having  a  direct 
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impact  on  tbe  trainee;  these  incentives  tended  to  be  more  costly  and  had  low  ad¬ 
ministrative  feasibility.  The  authors  proposed  four  experimental  social  incentive 
systems,  of  which  two  were  individual  incentive  systems  and  two  were  group  in¬ 
centive  systems.  Within  each  of  these  categories,  the  incentive  systems  were  di¬ 
vided  into  systems  administrated  by  the  training  instructor  or  by  tbe  instructor 
and  the  class.  The  rationale  for  proposing  group  incentives  was  that  perfor¬ 
mance  in  training  was  felt  to  be  affected  by  the  development  of  cohesiveness  in 
the  class  and  by  class  members  helping  each  other  reach  performance  standards. 

Several  researchers  have  reported  on  the  use  of  incentives  to  increase 
performance  in  the  military  (Cassileth,  1969;  Datel,  1970;  Datel  &  Legters,  1970- 
1971;  Dockstader,  Nebeker,  &  Shumate,  1960;  Dockstader,  Nebeker,  Nocella,  & 
Shumate,  1960;  Korman,  Glickman,  &  Prey,  1961;  Pritchard,  Von  Bergen,  & 
DeLeo,  1974).  In  general,  the  results  demonstrate  that  the  effectiveness  of  in¬ 
centives  is  highly  variable.  Korman,  et  al.  (1961)  found  that  disposition  to  enlist 
was  not  related  to  the  amount  of  the  incentive.  Pritchard,  et  al.  (1974)  found 
that  high- feasibility  incentives  (letters  of  commendation,  time-off,  avoidance  of 
work  details,  choice  of  uniform,  and  avoidance  of  marching  in  formation)  were 
not  effective  in  increasing  performance.  Pritchard,  et  al.  also  found  that  finan¬ 
cial  incentives  were  effective  only  with  one  of  their  two  experimental  groups. 
Cassileth  (1969)  found  that  incentives  of  3-day  passes  and  free  movies  were  ef¬ 
fective  only  in  increasing  performance  for  high  ability  trainees;  they  had  no  ef¬ 
fect  for  low  ability  trainees. 

These  results  from  the  military  setting  are  consistent  with  those  of  a 
meta-analysis  of  intervention  programs  which  found  that  financial  incentives  have 
a  broad  range  of  effects  (Guzso,  Jette,  &  Katzell,  1965).  These  authors  re¬ 
ported  an  average  effect  size  of  .57  standard  deviations  (d-statistic)  with  a 
broad  confidence  interval  that  also  included  zero  (-.10  to  1.24).  They  concluded 
that  "the  effects  of  incentive  programs  depend  heavily  on  the  circumstances  and 
methods  of  applying  them  and  ...  on  toe  criterion  of  productivity"  (p.  265). 
When  the  criteria  of  productivity  were  broken  down  into  output  (which  included 
quantity  and  quality  of  production),  withdrawal,  and  work  disruption,  toe  effect 
size  of  financial  incentive  programs  was  significantly  different  from  zero  for 
the  output  criteria  only. 


it  is  dear  from  toe  literature  tost  feedback  can  have  a  positive  effect  on 
individual  performance.  What  literature  exists  also  indicates  tort  feedback  can 
aahance  group  performance.  However,  moot  of  this  research  has  dealt  with  jobs 
that  were  quite  simple.  Usually,  everyone  did  toe  some  work,  everyone  worked 
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independently,  and  there  were  one  or  two  easily  collectible  objective  measures  of 
performance.  When  we  apply  these  findings  to  Air  Force  environments,  there  is 
a  very  important  difference.  The  vast  majority  of  Air  Force  jobs  are  much 
more  complex  than  those  documented  in  the  literature.  This  complexity  exists 
because  (a)  these  jobs  involve  individuals  doing  many  different  things,  (b) 
coordination  among  individuals  is  a  crucial  part  of  the  jobs,  and  (c)  objective 
measures  are  not  generally  available. 

This  suggests  that  to  use  feedback  techniques  in  Air  Force  environments, 
we  must  develop  ways  of  providing  feedback  for  these  complex  jobs.  This 
means  that  we  must  first  develop  measures  of  productivity  that  are  applicable  to 
complex,  interdependent  jobs  and  combine  them  with  techniques  of  giving  feedback 
that  are  effective  for  such  jobs. 

The  same  general  line  of  reasoning  applies  to  goal  setting  and  incentives. 
Both  can  be  effective.  But  it  is  not  clear  whether  the  positive  effects  will  also 
occur  in  a  military  setting  for  the  type  of  complex  jobs  typically  found  in  Air 
Force  organizations. 

Organizational  Productivity 

Although  much  has  been  written  on  the  subject  of  organizational  productiv¬ 
ity,  there  is  little  consensus  concerning  its  definition  (Tuttle,  1963).  This  lack 
of  consensus  is  perhaps  not  surprising,  since  there  are  many  approaches  to  and 
perspectives  on  productivity.  However,  several  major  issues  are  cited  throughout 
this  literature,  and  these  will  be  addressed  here. 

The  first  issue  to  be  addressed  is  whether  an  efficiency  or  an  ef¬ 
fectiveness  approach  should  be  used  in  measuring  productivity.  Both  have  been 
proposed  and  used.  Efficiency  is  typically  thought  of  as  an  output-to-input  ratio. 
For  example,  monthly  manufacturing  output  divided  by  manpower  used  to  produce 
that  output  would  be  an  efficiency  measure.  Effectiveness  is  usually  defined  as 
the  relationship  of  outputs  to  some  standard  or  expectation.  For  example, 
monthly  manufacturing  output  expressed  as  a  percentage  of  the  goal  for  that 
month  would  be  an  effectiveness  measure.  In  addition,  effectiveness  usually  in¬ 
cludes  quality  of  the  output  as  well  as  quantity. 

Efficiency  is  the  more  widely  used  of  the  two  concepts  since  it  is  easier 
to  measure  and  standardize  across  organizations,  industries,  and  nations  (Norman 
&  Bahiri,  1972).  When  we  hear  that  productivity  growth  in  the  United  States  has 
declined  over  the  last  20  years  (American  Productivity  Center,  1961),  it  is  an  ef¬ 
ficiency  ratio  that  is  being  quoted  (i.e.,  price  deflated  gross  national  product 
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divided  by  worker  hours).  Effectiveness  is  a  much  broader  concept  because  it 
includes  other  factors  such  as  standards,  objectives  of  the  organization,  expecta¬ 
tions  of  interested  parties  (e.g.,  shareholders,  regulatory  agencies,  and  cus¬ 
tomers),  and  the  viability  of  the  organization  relative  to  its  competition.  Pro¬ 
ponents  of  the  effectiveness  concept  argue  that  as  the  complexity  and  ambiguity 
of  the  work  increase,  effectiveness  measures  become  more  important  than  effi¬ 
ciency  measures  (Balk,  1975).  Some  authors  define  productivity  as  a  combina¬ 
tion  of  efficiency  and  effectiveness  (Balk,  1975;  Coulter,  1979;  Hanes  &  Kriebel, 
1971;  National  Center  for  Productivity  and  Quality  of  Working  Life,  1978;  Sib- 
son,  1976;  Tuttle,  1981). 

A  second  issue  is  what  perspective  should  be  taken  in  measuring  produc¬ 
tivity.  The  literature  indicates  that  the  approach  to  be  used  in  measuring  pro¬ 
ductivity  is  determined  by  the  perspective  of  those  doing  the  measuring.  Tuttle 
(1963)  suggests  five  perspectives:  those  of  the  economist,  engineer,  accountant, 
manager,  and  industrial/organizational  psychologist.  These  approaches  differ 
primarily  in  what  they  are  trying  to  learn  from  the  productivity  measurement. 
To  the  economist,  for  example,  productivity  is  output  divided  by  associated  inputs 
such  as  labor,  capital,  intermediate  products  purchased,  and  time.  This  approach 
is  typically  applied  to  very  macro  units  such  as  whole  industries  or  countries  to 
measure  the  economic  health  of  those  units.  The  industrial/organizational  psy¬ 
chologist  approach  focuses  on  the  the  personnel  subsystem  of  the  organization, 
and  its  measures  deal  with  the  efficiency  or  effectiveness  of  the  personnel. 
These  approaches  are  quite  different.  They  measure  different  things,  and  they 
are  used  for  different  purposes.  They  would  also  result  in  very  different  pro¬ 
ductivity  measurement  systems. 

The  next  issue  is  what  measures  should  be  included  in  the  measure  of 
productivity.  Clearly,  the  different  perspectives  such  as  the  economist's  and  the 
accountant’s  have  implications  for  what  measures  are  included.  There  are,  how¬ 
ever,  a  variety  of  other  possibilities.  Campbell  (1977),  for  example,  listed  30 
types  of  measures  that  have  been  used.  These  included  the  expected  measures 
such  as  effectiveness,  efficiency,  profit,  and  quality,  but  also  included  measures 
such  as  absenteeism,  growth,  morale,  control,  internalization  of  organizational 
goals,  evaluations  by  external  entities,  and  stability.  As  another  example, 
Seashore  and  Yuchtman  (1967)  reported  a  factor  analysis  of  organizational  pro¬ 
ductivity  scores  for  insurance  agencies.  They  identified  ten  factors,  many  of 
which  were  quite  different  from  those  listed  by  Campbell.  They  included  new 
member  productivity,  youthfulness  of  members,  business  mix,  manpower  growth, 
and  market  penetration. 
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The  variety  of  measures  that  could  be  included  in  a  productivity  measure¬ 
ment  system  clearly  shows  that  there  is  no  one  set  of  measures  that  best  defines 
productivity.  The  diversity  of  possible  measures  must  be  considered  in  the  de¬ 
sign  of  a  productivity  measurement  system. 

Apart  from  the  issue  of  what  productivity  is,  there  is  considerable  agree¬ 
ment  that  a  productivity  measurement  system  should  include  all  important  aspects 
of  the  organization's  work.  If  the  system  is  not  complete,  it  could  easily  en¬ 
courage  neglect  by  the  organization's  personnel  of  those  organizational  objectives 
which  are  not  part  of  the  measurement  system.  In  such  a  situation,  the  actual 
overall  effectiveness  of  the  organization  would  suffer  (Duerr,  1974). 

Another  broad  issue  is  whether  or  not  the  productivity  measurement  system 
should  provide  an  overall  index  of  productivity.  The  use  of  a  single  index  is  be¬ 
lieved  to  be  very  important  because  of  its  motivational  value.  A  single  index 
provides  the  members  of  the  unit  with  a  sense  of  productivity  improvement  or 
decrement.  The  single  index  would  also  seem  beneficial  for  management 
information  purposes.  A  large  number  of  pieces  of  information  on  organizational 
functioning  can  be  very  difficult  to  assimilate  and  use  for  making  decisions. 
Many  approaches  to  measuring  productivity  use  a  single  index  (e.g.  Felix  &  Riggs, 
1963;  Joint  Financial  Management  Improvement  Program,  1976;  Kim,  1980; 
Peeples,  1978;  Tuttle  &  Weaver,  1966a,  1966b;  Tuttle,  Wilkinson,  &  Matthews, 
1985). 

Conclusions  from  the  Productivity  Literature 

In  summary,  there  are  many  approaches,  perspectives,  and  issues  relevant 
to  productivity  measurement.  It  is  tempting  to  ask:  What  is  the  best  definition 
and  perspective  to  use  in  conceptualizing  productivity?  However,  we  believe  that 
this  is  the  wrong  question.  Efficiency  and  effectiveness  approaches  both  have 
their  place,  as  do  the  different  perspectives.  How  one  resolves  some  of  the 
other  issues  such  as  whether  to  use  a  single  index  of  productivity  and  what  to 
measure  depends  on  the  circumstances.  The  better  question  is:  Under  which  cir¬ 
cumstances  is  which  approach  most  appropriate? 

In  order  to  answer  this  question,  we  must  determine  the  purpose  of  mea¬ 
suring  productivity  in  the  Air  Force.  The  purpose  will  help  determine  the  cor¬ 
rect  approach.  For  different  purposes,  very  different  approaches  would  be  used. 


Our  Approach  To  Organizational  Productivity 


In  the  simplest  terms,  our  purpose  in  measuring  productivity  is  to  be  able 
to  increase  it  within  a  given  Air  Force  organization  or  part  of  that  organization. 
It  is  our  assumption  that  the  people  in  the  organization  have  a  great  impact  on  the 
productivity  of  the  organization.  Although  the  technical  subsystem  is  also  impor¬ 
tant,  our  focus  is  not  on  that  part  of  the  system  directly  but  rather  on  how  the 
technical  subsystem  is  used  by  the  people.  Our  position  is  that  to  increase  pro¬ 
ductivity  we  need  to  increase  the  productivity  of  the  people  in  the  organization 
through  increased  motivation.  With  increased  motivation,  personnel  would  exert 
more  effort  and  be  more  persistent  in  their  efforts;  they  would  work  more 
efficiently  in  the  sense  that  their  efforts  would  be  more  directly  related  to 
organizational  objectives;  and  they  would  improve  their  work  strategies  and  would 
use  their  own  and  others’  time  and  efforts  with  less  waste. 

Although  we  believe  that  both  efficiency  and  effectiveness  approaches  should 
be  included  in  a  productivity  measurement  system,  we  believe  that  the  appropriate 
approach  for  an  Air  Force  productivity  measurement  system  is  to  first  consider 
productivity  as  effectiveness  rather  than  efficiency.  We  take  this  position  for 
three  reasons.  First,  effectiveness,  with  its  orientation  toward  goal  attainment, 
is  a  broader  definition  of  productivity,  since  it  results  in  a  measurement  system 
that  expresses  productivity  in  terms  of  how  good  that  productivity  is.  In 
contrast,  an  efficiency  approach  does  not  carry  with  it  evaluative  information  on 
what  is  a  good  or  bad  level  of  efficiency.  The  second  reason  for  adopting  the 
effectiveness  approach  is  that  this  approach  can  more  easily  generate  a  mea¬ 
surement  system  that  combines  all  aspects  of  the  organization’s  productivity  into 
a  single  measure.  The  final  reason  is  that  we  believe  this  approach  makes  it 
possible  to  get  an  effectiveness  measure  and  weight  it  by  inputs  to  get  a  system 
that  combines  the  best  aspects  of  both  the  effectiveness  and  the  efficiency  ap¬ 
proaches. 

Given  our  purpose  of  increasing  productivity  in  the  Air  Force,  it  is  criti¬ 
cal  that  the  measurement  system  be  complete  so  that  increases  in  measured  as¬ 
pects  of  the  work  are  not  made  at  the  expense  of  equally  important  but  unmea¬ 
sured  aspects.  Finally,  the  individual  measures  should  be  combined  into  an 
overall  measure  of  productivity  for  both  motivational  and  informational  purposes. 
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Research  Questions 


Having  reviewed  the  literature  and  settled  upon  an  appropriate  approach, 
we  proceeded  toward  our  specific  research  objective,  which  was  to  provide  an¬ 
swers  to  the  following  questions: 

1.  Can  the  productivity  measurement  system  be  effectively  developed  in 
an  Air  Force  organization? 

2.  Will  giving  feedback  with  the  productivity  measurement  system  in¬ 
crease  productivity? 

3.  Will  goal  setting  and  incentives  increase  productivity  over  feedback? 

4.  Will  using  the  system  change  work  attitudes? 

5.  Will  using  the  system  be  seen  as  valuable  by  the  users? 

6.  Will  the  system  operate  successfully  after  the  departure  of  the  re¬ 
search  team? 

m.  METHOD 

Site 


The  present  effort  was  conducted  at  an  Air  Force  base  in  the  southwest¬ 
ern  United  States,  and  involved  sections  in  Maintenance  and  in  Resource  Man¬ 
agement.  The  maintenance  unit  was  the  Communications  and  Navigation  section 
(Comm/Nav)  in  the  Component  Repair  Squadron.  There  were  four  sections  in 
Resource  Management,  which  together  comprised  the  Materiel  Storage  and  Distri¬ 
bution  Branch  (MS&D)  of  the  Supply  Squadron.  The  four  sections  were  Receiv¬ 
ing,  Storage  and  Issue,  Pickup  and  Delivery,  and  Inspection. 

The  Comm/Nav  section  repaired  a  variety  of  electronic  equipment  used  for 
aircraft  communication  and  navigation.  The  number  of  personnel  in  Comm/Nav 
ranged  from  29  to  35  during  the  course  of  the  project.  The  MS&D  branch  was 
essentially  the  base  warehouse.  Property  was  delivered  to  the  warehouse  and 
checked  in  by  the  Receiving  section.  Storage  and  Issue  shelved  the  property  and 
retrieved  it  as  it  was  ordered  by  units  on  the  base.  The  Delivery  section  deliv¬ 
ered  the  property  to  units  on  base  that  had  ordered  it.  Inspection  made  sure  the 
property  was  in  good  condition,  and  ensured  that  regulations  were  being  followed 
concerning  property  packaging,  storage,  and  identification.  The  number  of  per¬ 
sonnel  in  the  MS&D  branch  ranged  from  47  to  54  during  the  course  of  the  pro¬ 
ject,  with  Receiving  averaging  15,  Storage  and  Issue  15,  Pickup  and  Delivery  13, 
and  Inspection  7. 


The  theoretical  background  for  this  approach  to  the  measurement  of  orga¬ 
nizational  productivity  stems  from  the  theory  of  organizational  behavior  presented 
by  Naylor,  Pritchard,  and  Ilgen  (1980).  In  this  theory,  an  individual's  role  is 
seen  as  a  series  of  relationships,  called  contingencies.  These  contingencies  not 
only  indicate  the  important  things  that  the  person  is  expected  to  do  in  the  job,  but 
also  show  the  relationship  between  the  amount  of  each  of  these  activities  and 
how  that  amount  is  evaluated. 


This  approach  to  roles  has  the  advantage  of  indicating  more  than  the  typi¬ 
cal  information  present  in  role  specification.  The  typical  information  is  limited 
to  a  listing  of  the  important  duties  to  be  performed  on  the  job.  In  the  Naylor, 
Pritchard,  and  Ilgen  approach,  this  information  is  supplemented  by  what  level  of 
performance  is  expected  in  each  area,  and  how  positively  or  negatively  each  level 
of  performance  is  evaluated. 

In  essence,  we  used  the  Naylor,  Pritchard,  and  Ilgen  conceptual  approach 
of  products  and  contingencies,  and  extended  its  application  from  individuals  to 
organizational  units.  This  application  led  to  the  development  of  a  number  of 
unique  features  for  a  productivity  measurement  system.  We  shall  discuss  these 
later  in  the  report. 

A  second  source  for  the  development  of  our  approach  was  the  work  of 
Tuttle  (1981;  Tuttle  et  al.,  1985;  Tuttle  &  Weaver,  1986a,  1966b).  In  this  work, 
also  supported  by  the  Air  Force  Human  Resources  Laboratory,  Tuttle  developed 
an  approach  to  measuring  productivity  which  included  methods  of  going  from  what 
we  call  products  to  obtaining  objective  indicators  of  how  well  these  products 
were  provided.  He  used  a  variety  of  group  techniques  to  develop  products  and 
indicators,  some  of  which  we  also  used. 

Steps  in  the  Development  of  the  Productivity  Measurement  System. 

The  technique  used  to  generate  the  productivity  measurement  system  is  de¬ 
scribed  more  fully  in  Pritchard,  et  al.  (1987a),  but  will  be  summarized  here.  It 
consists  of  four  distinct  steps:  (a)  identify  salient  products,  (b)  develop  indica¬ 
tors  of  these  products,  (c)  establish  contingencies,  and  (d)  put  the  system  to¬ 
gether. 
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Step  1:  Identify  Products. 


Every  organization  has  a  set  of  activities  that  it  is  expected  to  perform. 
These  activities  result  in  a  set  of  what  Naylor,  et  al  (1980)  called  products.  In 
using  the  term  "product,"  we  mean  more  than  merely  a  tangible  thing  that  is  pro¬ 
duced.  Products  can  be  thought  of  as  the  set  of  objectives  that  the  organization 
is  expected  to  accomplish.  The  productivity  of  the  organization  is  a  function  of 
how  effectively  the  organization  generates  these  products.  The  first  step  in  de¬ 
veloping  the  productivity  measurement  system  is  to  identify  these  products. 

To  present  the  steps  involved  in  developing  the  productivity  measurement  sys¬ 
tem,  we  shall  use  an  extended  hypothetical  example  that  will  make  each  step 
more  concrete.  For  this  example,  we  shall  use  an  organization  similar  to  our 
Comm/Nav  section.  Assume  that  the  unit  diagnoses  and  repairs  electronic  equip¬ 
ment.  Personnel  are  responsible  for  the  repair  of  various  items  that  have  mal¬ 
functioned  and  have  been  brought  to  the  shop  for  repair.  Their  primary  respon¬ 
sibility  is  to  repair  these  items  as  quickly  and  as  accurately  as  possible.  If  they 
repair  an  item  and  it  does  not  function  properly,  it  is  returned  for  reaccom¬ 
plishment  of  the  repair.  They  are  periodically  inspected  by  a  Quality  Control 
function,  which  determines  whether  they  are  accurately  following  the  procedures 
for  repair  that  are  detailed  in  available  repair  manuals.  The  maintenance  unit 
also  has  responsibility  for  conducting  on-the-job  training,  and  a  person  can  repair 
a  piece  of  equipment  only  if  he/she  has  passed  the  training  certification  required 
for  that  piece  of  equipment.  Thus,  it  is  important  that  a  sufficient  number  of 
people  be  qualified  through  training  so  that  all  the  items  can  be  repaired  in  a 
timely  manner. 

To  develop  the  system,  the  first  step  would  be  to  meet  with  people  from  the 
organization  to  identify  the  salient  products.  Let  us  assume  that  the  following 
products  are  identified: 

1.  Quality  of  repair. 

2.  Ability  to  meet  demand  for  repairing  items  (i.e.,  the  organization’s 
ability  to  repair  the  needed  equipment  quickly). 

3.  Ability  to  meet  training  needs  (i.e.,  the  degree  to  which  the  organi¬ 
zation  meets  its  on-the-job  training  needs). 

In  actual  fact,  there  might  well  be  more  products  in  such  an  organization. 
However,  since  our  intent  here  is  to  explain  the  logic  of  the  system,  we  shall 
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use  only  these  three  so  that  die  example  remains  simple  enough  for  dear  pre¬ 
sentation. 

Step  2:  Develop  Indicators. 

Once  the  products  are  determined,  the  next  step  is  to  develop  indicators  for 
each  of  these  products.  An  indicator  is  a  measure  of  how  well  the  organisation 
is  generating  the  product  in  question.  The  indicators  are  determined  through  in¬ 
teraction  with  the  people  in  the  organization,  who  are  asked  to  think  of  those 
things  which  would  show  how  well  people  in  die  organization  are  producing  their 
products.  There  may  be  only  one  indicator  for  a  given  product,  or  there  may  be 
more  than  one.  Some  indicators  will  already  be  available;  some  will  have  to  be 
newly  developed.  After  die  indicators  are  discussed  and  refined,  the  products 
and  indicators  might  look  something  like  this: 

Product  1.  Quality  of  repair. 

Indicator  A:  Return  rate:  percentage  of  items  returned  for 

reaccomplishment  of  repair. 

Indicator  B:  Percentage  of  Quality  Control  inspections  passed. 

Product  2.  Ability  to  meet  demand  for  repairing  items. 

Indicator:  Number  of  units  repaired  divided  by  total  number 

of  units  brought  in  for  repair. 

Product  3.  Ability  to  meet  training  needs. 

Indicator:  Number  of  people  qualified  to  work  on  each  type 

of  item  to  be  repaired,  divided  by  the  number  of 
people  needed  to  be  qualified. 

As  mentioned  above,  this  would  not  be  a  complete  list  of  products  and  in¬ 
dicators  for  such  an  organization,  but  it  does  serve  to  explain  the  concept  used 
for  the  productivity  measurement  system. 

Step  3.  Establish  Contingencies. 

Once  the  products  and  indicators  have  been  identified  and  approved,  the  next 
step  is  to  establish  the  contingencies.  A  contingency  is  the  relationship  between 
the  amount  of  the  indicator  and  the  effectiveness  of  that  amount  of  the  indicator. 
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Fifure  1  preeente  an  oaapto  of  «  contiagaacy.  The  top  half  of  the  figure 
shows  the  general  form  of  a  cotfagmqr.  Tim  horlmml  axis  represents  the 
smoimt  of  the  Uteator,  naghg  from  the  worst  possible  level  to  the  hem  possi¬ 
ble  level.  The  vertical  erie  of  the  figure  rimers  the  effect! veneea  vehms  of  the 
various  levels  of  rim  isriirator.  h  ranges  from  4100*.  which  is  maximum  effec¬ 
tiveness,  to  -100.  arisen  effectiveness,  ft  also  has  a  aero  point  which  is  de¬ 
fined  as  rim  expected  or  neutral  level  of  effectiveness.  That  is,  the  aero  point 
in  neither  positive  aor  negative.  For  this  wimple,  we  have  choc  an  the  first  in- 
dteetor:  return  rate.  Assume  that  the  people  in  the  organisation  indicate  that  the 
bant  that  ie  ever  possible  it  to  have  e  2%  return  rate.  They  tee  it  ee  impossible 
to  do  better  then  2%  because  approximately  2%  of  the  electronic  components  they 
use  for  repairs  can  work  properly  when  installed  end  checked,  but  fail  almost 
immediately  when  put  into  use.  Let  us  assume  they  say  that  the  worst  possible 
return  rate  would  be  TOIL 

Once  the  best  and  worst  possible  levels  of  productivity  have  been  established 
by  the  organisational  personnel,  the  next  task  is  to  determine  the  zero  point;  that 
is,  the  expected  level,  the  level  that  ie  neither  especially  good  nor  especially  bad 
ia  terms  of  productivity.  Once  trie  it  established,  a  point  would  be  placed  on 
dm  figure  at  the  iatersecthm  of  rim  aero  point  of  the  vertical  axis  and  the  level 
of  neutral  point  an  rim  horisontnl  axis.  For  example,  if  the  neutral  point  was 
idautified  as  a  return  rate  of  10%,  it  would  be  indicated  as  shown  in  die  bottom 
half  of  Figure  1. 

Next,  the  maximum  and  minimum  effectiveness  levels  for  rim  indicators 
would  be  established.  The  first  step  ie  to  list  the  maximums  for  each  of  the 
indicators.  The  group  of  incumbents  and  supervisors  is  then  asked  to  rank  or¬ 
der  these  maximums  ia  terms  of  the  contribution  of  each  to  the  overall  effec¬ 
tiveness  of  the  unit.  The  group  discusses  trie  and  consensus  is  reached.  The 
maximum  with  rim  highest  importance  rank  is  then  given  an  effectiveness  value 
of  4-100,  and  rim  group  is  aakad  to  rate  the  other  maximums  as  percentages  of 
tfm  4-100  maximum.  For  example,  if  the  maximum  of  a  given  indicator  were  only 
half  ns  important  to  dm  effectiveness  of  the  unit  as  the  most  important  (best) 
maximum,  it  would  be  given  a  value  of  450.  An  analogous  process  is  performed 
for  the  minimum  values  of  rim  indicators,  except  die  most  important  (worst) 
minimum  ia  not  constrained  to  a  value  of  -100.  It  is  given  rim  value  that  the 
group  feels  ia  appropriate. 


Ones  rim  sera  prints  hues  been  identified  and  rim  effectiveness  values  of 
rim  mwimnma  and  niainnma  established,  rim  remainder  of  the  points  in  the 
function  for  each  indicator  are  developed  by  rim  group.  Group  discussion  is  con¬ 
tinued  until  consensus  is  reached. 
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Assume  that  the  pmoad  la  the  orpdaiiai  Mid  that  return  rata  »u  an 
important  —part  of  their  work,  (hat  to  be  at  the  wiaiasuw  would  correspond  to 
an  effectiveness  of  -80,  aad  to  be  at  the  maxtauaa  would  ba  a  +70.  Aftar  tke 
otker  potato  oa  tka  fluctta  ara  tan Hflad,  ffeta  ari^K  roatat  te  a  contingency  inch 
aa  tkat  akowa  ta  tka  hottam  of  Figure  1.  k  tadtaataa  tkat  going  above  tka  aoutral 
potat  of  a  10K  ratara  rata  la  positive,  but  changes  ara  aot  liaaar,  ta  tkat  oace 
ratura  rate  darraaaaa  to  6ft,  further  darraaaaa  do  aot  represent  m  Croat  aa  ta- 
croaoa  ta  effectiveness.  Ukawtaa,  at  tea  taw  tad,  oaca  tka  ratura  rata  gate  aa 
bad  aa  14ft,  they  ara  doing  vary  badly  aad  tawar  rataa  ara  proporttaaally  aot  aa 
bad. 


Aftar  ttaa 


aU 


procaaa  haa  baaa  oooaplatad  for  aack  of  tka  tadicatora,  aad  ooce 
acalad  aad  reviewed  for  accuracy,  tka  contingency  aat  would  be 


A  aaapk  coottagaacy  aat  ia  praaaotad  ta  Figure  2.  For  each  indicator,  there 
in  a  noattaganry  with  ita  ataattaaaa,  auauaum,  aad  expected  level  or  zero  potat. 
aad  a  fuacttaa  relattag  it  to  affect! vaaosa.  The  first  contingency,  that  for  return 
rata,  te  the  aaaM  aa  teat  akowa  ta  tka  tawar  part  of  Figure  1.  The  second  coo¬ 
ttagaacy  ta  for  tea  percent  of  Quality  Control  taapoctioos  passed.  Note  that  for 
this  coattagaacy  tka  expected  level  ia  tkat  100ft  of  these  inspections  be  passed. 
Rectal  that  theao  iaaparttaaa  ara  aot  inspections  of  the  final  work  but  rather,  in¬ 
spections  of  the  procaaa  tka  technician  goes  through  ta  doing  the  repair.  It  is  an 
index  of  kow  well  tke  parson  ta  following  the  manual  ta  doing  the  repairs.  It  is 
expected  tkat  all  repairs  will  ba  done  ta  accordance  with  the  manual.  Thus,  this 
coattafaacy  shows  tkat  the  expected  level  is  doing  all  repairs  (100ft)  in  accor¬ 
dance  with  tke  amenta.  Since  it  ia  not  possible  to  pass  more  than  100ft  of  in¬ 
spections,  there  are  no  positive  values.  Thus,  anything  lass  than  this  is  below 
expectations,  aad  results  ta  negative  effectiveness.  Together,  these  two  coatin- 
gaadoa  cover  tka  product  of  duality  of  repair. 

Tka  second  product  haa  only  one  indicator,  aad  bases  only  one  contingency. 
The  prodart  ia  ability  ta  asset  repair  demand,  and  the  indicator  is  the  number  of 
units  repaired  divided  by  dm  number  brought  ta,  expressed  as  a  percentage.  This 
rnatingsary  to  roughly  linear,  except  tkat  a  value  within  dm  90ft  to  100)1  range 
rapraaeate  a  amjor  jwnp  ta  affsrtivaaaee. 

Tke  last  product  to  masting  tretateg  needs,  aad  atae  has  only  one  indicator  - 
number  of  paepie  gnoliflsd  (throutai  training)  to  repair  ogtapmeat,  divided  by  the 
number  needed,  egproooad  as  a  percentage.  For  tkta  indicator  it  is  possible  to  go 

paopie  to  ba  qualified  to 
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repair  a  given  piece  of  equipment,  it  could  actually  have  more  than  IS.  How¬ 
ever,  the  contingency  become*  flat  after  110)1,  indicating  that  having  more  than 
110%  is  no  more  effective  than  having  110%.  The  idea  is  that  ooce  there  is  a 
small  excess  over  the  maximum  number  needed,  having  additional  trained  person¬ 
nel  is  not  important. 

Two  things  are  particularly  noteworthy  about  these  contingencies.  The  first 
is  that  the  overall  slope  of  the  function  expresses  the  relative  importance  of  the 
indicator.  For  example,  the  overall  slope  for  the  first  indicator  (return  rate)  is 
steeper  than  for  the  second  indicator  (percent  of  inspections  passed).  This  re¬ 
flects  the  fact  that  although  it  is  important  to  pass  inspections  which  show  that 
the  process  of  doing  the  repair  was  accurate,  actually  doing  the  repair  so  that 
the  item  functions  properly  is  more  important.  Second,  the  contingencies  can  be 
non-linear.  As  shall  be  discussed  below,  this  is  necessary  to  accurately  reflect 
the  realities  of  an  organization’s  functioning.  In  many  cases,  the  relationships 
that  actually  exist  are  simply  not  linear. 

It  is  important  to  recognize  what  these  properties  of  the  contingencies  do. 
First,  the  contingencies  reflect  differential  importance.  Different  things  the 
organization  does  are  not  equally  important.  Aspects  of  the  work  that  are  very 
important  get  steeper  contingencies  than  aspects  that  are  less  important.  For  ex¬ 
ample,  in  Figure  2,  the  first  indicator  for  quality  of  repair  has  a  range  in  ef¬ 
fectiveness  from  -80  to  +70,  whereas  meeting  training  needs  ranges  from  -60  to 
+10.  This  indicates  that  quality  of  repair  is  more  important  than  meeting  train¬ 
ing  needs  since  variations  in  quality  of  repair  have  a  greater  impact  on  the  ef¬ 
fectiveness  of  the  organization.  Thus,  the  relative  importance  of  each  aspect  of 
the  work  is  incorporated  into  the  contingencies. 

Secondly,  contingencies  allow  for  the  existence  of  non-linearities  between 
amount  of  the  measure  and  productivity.  A  linear  relationship  would  mean  that  to 
improve  a  given  amount  at  the  low  end  of  the  measure  would  be  as  good  as  im¬ 
proving  that  same  amount  at  the  high  end.  However,  in  the  real  world  it  is  very 
common  for  values  in  the  middle  range  of  an  indicator  to  represent  large  im¬ 
provements  in  productivity,  and  values  at  the  high  end  to  represent  a  point  of  di¬ 
minishing  returns.  That  is,  ooce  an  organization  gets  to  a  fairly  high  level  of 
productivity  on  one  aspect  of  the  work,  it  is  frequently  better  to  try  to  improve 
something  that  they  are  not  doing  as  well,  rather  than  continue  to  improve  some¬ 
thing  that  is  already  at  a  high  level. 

For  example,  if  the  repair  shop  were  operating  with  a  very  low  return 
rate,  it  might  be  better  to  try  to  improve  meeting  training  needs  rather  than  at¬ 
tempting  to  further  improve  its  return  rate.  Thus,  evea  though  return  rate 


27 


overall  is  more  important  than  training,  if  return  rate  is  good,  improving  a  low 
degree  of  training  readiness  can  become  more  important  to  the  overall  effective¬ 
ness  of  dm  organisation.  Another  example  of  this  non-linearity  would  be  a  situ¬ 
ation  such  as  that  depicted  for  training  needs.  Once  the  organization  reaches  a 
certain  point,  further  increases  are  not  more  effective  since  all  the  people  that 
are  necessary  are  already  trained.  The  contingencies  in  our  system  capture  this 
non-linearity  and  thus  provide  a  more  accurate  picture  of  the  organisation^  func¬ 
tioning. 


Step  4.  Put  the  System  Together. 

Once  the  contingencies  are  completed  and  approved  by  management,  the  last 
step  is  to  put  the  system  together.  This  would  be  accomplished  by  first  collect¬ 
ing  the  indicator  data  for  a  given  period  of  time.  If  the  time  period  selected 
were  a  month-long  period,  the  data  for  the  four  indicators  would  be  collected  at 
the  end  of  dm  month.  Then,  based  on  the  contingencies,  effectiveness  scores 
would  be  determined  for  each  indicator  by  calculating  the  effectiveness  for  that 
level  of  the  indicator.  This  is  illustrated  in  Figure  3.  For  example,  if  the 
maintenance  unit  had  a  return  rate  of  6%  in  the  month  of  March,  examining  the 
contingency  indicates  that  such  a  return  rate  is  associated  with  an  effectiveness 
score  of  +60  (i.e.,  a  value  of  6%  return  rate  on  the  horizontal  axis  is  associated 
with  an  effectiveness  value  of  +60  on  die  vertical  axis).  Continuing  this  process 
would  give  an  effectiveness  value  for  each  indicator,  as  exemplified  in  the  Fig¬ 
ure  3. 

Once  the  effectiveness  values  are  determined,  they  can  he  summed  to  obtain 
the  overall  effectiveness  score  for  products  with  more  than  one  indicator,  as 
seen  for  the  first  product.  The  total  effectiveness  of  the  product  Quality  of  Re¬ 
pair  would  he  the  sum  of  the  two  indicators  comprising  that  product:  +60  for 
return  rate  end  -10  for  qualify  control  inspections  passed,  for  a  total  of  +60. 
Next,  overall  productivity  can  he  calculated  by  summing  all  of  the  effectiveness 
scores,  b  the  example,  this  Overall  Effectiveness  score  is  +30. 

These  effectiveness  scores  have  a  distinct  manning,  in  that  a  score  of 
zero  moans  that  dm  organ! gatinn  is  masting  exportations;  that  is,  their  productiv¬ 
ity  is  neither  particularly  good  nor  had.  As  the  score  becomes  positive,  they  are 
exceeding  expectations.  The  more  positive  the  score,  dm  more  they  are  exceed¬ 
ing  expectations.  As  the  score  becomes  aagodve,  they  are  below  expectations. 
The  closer  they  ore  to  be  meedmnm  poooMe  overall  effectiveness  score,  the 
closer  they  ore  to  thair  hast  poosMs  prodoctfvity. 


Figure  3.  Completed  System 


PRODUCTIVITY:  MAINTENANCE  UNIT 
DATE:  March,  1967 


INDICATOR 
DATA:  MARCH 

EFFECTIVENESS 

SCORE 

I.  Quality  of  Repair 

A.  Return  Rate 

6% 

•tfO 

B.  Percent  Quality  Control 

95% 

-10 

Inspections  Passed 


Total  Effectiveness:  Quality  of  Repair  *  +50 


II.  Meeting  Repair  Demand 

A.  Percent  Demand  Met  90%  +10 

01.  Meeting  Training  Needs 

A.  Percent  Qualified/Need  80%  -40 


OVERALL  EFFECTIVENESS  -  +20 
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This  ability  to  simply  sum  effectiveness  scores  is  one  of  the  major  ad¬ 
vantages  of  the  system.  Because  the  contingencies  reflect  the  relative  importance 
and  the  non-linearity  of  the  indicators,  these  factors  are  already  incorporated  in 
the  system;  thus,  a  simple  summing  does  indeed  reflect  the  overall  effectiveness 
of  the  unit.  As  will  be  discussed  later,  this  property  also  makes  it  possible  for 
the  system  to  be  used  to  aggregate  across  individual  units  to  determine  the  pro¬ 
ductivity  of  larger  and  larger  units  of  the  organization. 

Quality  of  the  System 

In  order  for  this  approach  to  be  a  good  measure  of  organizational  produc¬ 
tivity,  it  is  obvious  that  it  needs  to  be  accurate.  This  means  several  things.  It 
means  that  the  listing  of  products  and  indicators  must  be  complete.  The  system 
can  easily  produce  a  situation  where  those  things  that  are  measured  are  attended 
to,  and  those  that  arc  not  measured  are  somewhat  ignored.  If  there  are  impor¬ 
tant  functions  of  the  unit  that  are  not  included  in  the  products,  or  if  important 
indicators  are  omitted,  this  uneven  attention  to  important  functions  can  have  very 
dysfunctional  consequences  for  the  organization.  In  addition  to  being  complete, 
the  system  must  accurately  reflect  what  the  unit  should  be  doing.  This  means 
that  the  products,  indicators,  and  contingencies  that  are  used  in  the  system  must 
be  the  correct  objectives,  must  be  accurate  measures,  and  must  be  correctly 
scaled  on  effectiveness. 

Both  accuracy  and  completeness  are  dealt  with  in  the  development  of  the 
system  by  having  a  clear  process  of  approval  of  the  system  at  higher  levels  of 
the  organization.  This  approval  process  is  made  clear  from  the  start.  That  is, 
at  the  beginning  of  the  development  of  the  system,  all  participants  are  told  that 
incumbents  and  supervisors  will  develop  the  products  and  indicators,  which  will 
then  be  presented  to  higher  management  for  approval.  Once  higher  management 
has  approved  the  products  and  indicators,  the  supervisory  groups  develop  the  con¬ 
tingencies,  which  must  also  be  formally  approved  by  higher  management.  Al¬ 
though  this  approval  mechanism,  and  the  multiple  inputs  that  it  provides,  does  not 
guarantee  completeness  and  accuracy  of  the  system,  it  provides  a  system  of 
checks  and  balances  so  that  the  system  will  be  as  complete  and  accurate  as  pos¬ 
sible. 


A  final  point  about  the  quality  of  the  system  is  that  the  development  of  the 
system  necessarily  introduces  subjectivity  into  the  system.  Subjectivity  is  present 
in  the  listing  of  the  products  and  indicators,  and  especially  in  the  ratings  that  are 
used  in  the  contingencies.  Subjectivity  is  present,  but  this  is  not  necessarily  a 
problem.  The  elements  of  the  system— products,  indicators,  and  contingencies— 
are  actually  statements  of  policy.  As  a  whole,  they  say  (a)  what  is  important  to 


the  functioning  of  the  unit,  (b)  the  level  of  output  that  is  expected  (the  zero 
point),  (c)  how  good  other  levels  of  output  are,  and  (d)  the  relative  importance 
of  different  types  of  functions  for  the  unit.  These  determinations  represent 
policy,  and  policy  is  a  subjective  thing.  A  manager’s  primary  responsibility  is  to 
set  policy,  in  the  sense  that  he/ she  must  determine  the  priorities  for  resource 
allocation  within  his/her  unit.  What  this  approach  does  is  to  reduce  ambiguity  in 
policy  and  priorities  by  formally  discussing  them,  quantifying  them,  and  subjecting 
them  to  formal  review  and  approval  by  the  management  of  the  organization. 

Priorities 


There  are  two  other  unique  features  of  the  productivity  measurement  sys¬ 
tem  that  should  be  described;  the  first  is  the  system’s  capability  to  generate  unit 
priorities. 

The  system  offers  a  way  to  develop  a  clear  set  of  priorities  for  improving 
productivity.  Recall  from  Figure  3  that  for  a  given  time  period  (e.g.,  a  month), 
the  system  presents  the  actual  amount  of  each  indicator  achieved  for  that  period, 
and  the  effectiveness  levels  of  those  amounts  of  the  indicators.  It  would  be  a 
simple  matter  to  look  at  the  contingency  for  each  indicator  and  calculate  the 
effectiveness  gain  that  would  occur  if  the  unit  went  up  one  increment  on  each  of 
the  indicators  during  the  next  period.  For  example,  if  the  unit  had  a  return  rate 
of  6%  in  March,  as  is  indicated  in  Figure  3,  for  them  to  go  to  the  next  level  up 
(a  4%  return  rate)  in  April  would  mean  an  increase  in  effectiveness  from  +60  to 
+65,  for  a  gain  in  effectiveness  of  +-5  units.  This  could  be  calculated  for  each 
indicator.  Once  it  was  calculated,  one  could  rank  order  the  changes  from  high¬ 
est  to  lowest  gain.  Such  a  listing  for  our  example  is  shown  in  Figure  4. 

This  information  communicates  exactly  what  should  be  changed  to  maxi¬ 
mize  productivity.  The  example  shows  that  the  best  thing  the  unit  can  do  is  to 
focus  their  efforts  on  meeting  repair  demand  if  they  want  the  highest  increase  in 
productivity.  Once  this  is  done,  or  if  it  is  not  possible  at  the  present  time,  the 
next  best  thing  they  could  do  is  to  improve  training  so  that  more  people  are 
qualified.  Improving  quality  control  inspections  and  improving  return  rate  are  the 
least  important  in  terms  of  increasing  productivity,  with  quality  control  inspec¬ 
tions  slightly  more  important. 

Thus,  the  system  can  generate  a  set  of  priorities  that  unit  personnel  can  use 
to  guide  their  efforts  to  increase  productivity.  This  would  aid  in  decisions  about 
resource  allocation,  and  where  to  foe  s  to  identify  barriers  to  productivity. 
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Priorities  for  Increasing  Productivity 


Figure  4. 


PRIORITIES  FOR:  APRIL.  1987 

CHANGE 

Percent  Demand  Met  from  90%  to  100% 

Percent  Qualified  (Training)  from  80%  to  90% 

Percent  Quality  Control  Inspections  Passed 
from  95%  to  100% 

Return  Rate  from  6%  to  4% 


GAIN  IN 

EFFECTIVENESS 


+90 


+20 


+10 

+5 


Aggregation  Across  Units 


A  second  unique  feature  of  the  productivity  measurement  system  is  the 
ability  to  aggregate  across  organizational  units.  It  is  quite  valuable  to  have  a 
productivity  measurement  system  for  a  given  unit,  or  several  units.  It  would  be 
even  more  valuable  if  one  could  aggregate  the  measurement  system  from  the  sev¬ 
eral  different  units  into  one  measure  applicable  across  all  the  units.  For  exam¬ 
ple,  if  a  branch  were  composed  of  four  separate  units,  it  would  be  valuable  to 
have  a  measure  for  each  unit,  and  to  be  able  to  combine  those  four  measures 
into  a  single  measure  for  the  entire  branch.  In  most  productivity  measurement 
systems  this  is  not  possible,  since  the  measurement  system  varies  from  unit  to 
unit.  An  advantage  of  our  approach  is  that  it  is  possible  to  do  such  across-unit 
aggregation.  Each  unit  is  measured  on  a  common  metric:  overall  effectiveness. 
The  method  for  combining  their  individual  effectiveness  values  to  determine  the 
overall  effectiveness  across  units  merely  involves  a  simple  additional  scaling  step 
in  the  development  of  the  contingencies;  this  procedure  is  explained  in  detail  in 
the  report  on  the  productivity  measurement  system  (Pritchard,  et  al.,  1987a). 

This  rescaling  process  has  the  effect  of  adjusting  the  effectiveness  scores 
of  the  different  units  in  the  branch  for  any  differences  in  importance  of  the  dif¬ 
ferent  units.  Once  it  is  finished,  the  overall  effectiveness  values  from  the  dif¬ 
ferent  units  can  simply  be  summed  to  determine  the  overall  effectiveness  of  the 
entire  branch.  This  approach  to  aggregation  can  be  extended  to  larger  and  larger 
units,  so  that,  if  desired,  a  single  index  of  the  productivity  of  the  entire  organi¬ 
zation  can  be  developed. 

System  Development 

To  actually  develop  the  productivity  measurement  system  at  the  base, 
meetings  were  held  jointly  with  incumbents  and  supervisors.  First  products  and 
then  indicators  were  developed.  Once  the  products  and  indicators  had  been  fi¬ 
nalized  by  group  consensus,  the  next  step  was  to  obtain  approval  of  the  products 
and  indicators  from  higher  management.  Meetings  were  held  with  higher  man¬ 
agement  where  the  products  and  indicators  were  presented,  discussed  in  detail, 
and,  with  some  revisions,  approved.  As  an  example,  the  final  set  of  products 
and  indicators  for  Comm/Nav  are  presented  in  Figure  5. 

The  next  step  was  the  development  of  the  contingencies.  Meetings  were 
again  held  with  incumbents  and  supervisors  of  each  unit.  First,  the  maximum 
and  minimum  indicator  levels  were  established,*  then  the  zero  points  were  gener¬ 
ated.  These  decisions  also  took  considerable  time,  resulted  in  considerable  ini¬ 
tial  disagreement,  and  were  done  over  multiple  meetings.  These  meetings  were 
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Figure  5.  Comm/Nav  Products  and  Indicators. 


Product  1.  Equipment  Repair 

Bounces:  Percentage  of  repaired  equipment  that  did  not  function 
immediately  after  installation. 

Percent  QA  (Quality  Assurance)  inspections  passed. 

AWM:  Number  of  units  awaiting  maintenance. 

AWP:  Number  of  units  awaiting  parts. 

Demand  Met:  Percentage  of  equipment  brought  in  for  repairs  that 
was  actually  repaired. 


Products  Training 

STS  Tasks  Completed:  Mean  number  of  standard  (more  basic) 
training  tasks  completed  for  personnel  in  training. 

Percent  Qual  Tasks  Completed,  Comm:  Mean  percent  of  advanced 
training  tasks  completed  for  personnel  repairing 
communications  equipment. 

Percent  Qual  Tasks  Completed,  Nav:  Mean  percent  of  advanced 
training  tasks  completed  for  personnel  repairing  navigation 
equipment. 

Scheduled  Training  Tasks  Overdue:  Total  number  non-technical 

(e.g.  military)  training  requirements  not  met  on  time  for  all 
shop  personnel. 


Product  3.  Other  Duties 


Mobility  Equipment:  Number  of  pieces  of  equipment  used  for 
mobility  exercises  that  were  not  calibrated  by  the  shop  on 
schedule. 

PMEL  Overdue:  Number  of  pieces  of  shop  calibration  and  test 
equipment  that  were  not  calibrated  by  the  shop  on  schedule. 

Percent  349  Errors:  Percent  of  errors  on  a  major  manpower 
documentation  form. 

Missed  Appointments:  Number  of  formal  on-base  appointments 
missed. 
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held  until  consensus  was  established.  The  contingencies  were  then  presented  to 
management  for  discussion  and  approval. 

T  reatments 


Once  the  productivity  measurement  system  was' completed,  the  next  phases 
of  the  research  involved  instituting  the  interventions  of  feedback,  goal  setting, 
and  incentives.  To  do  this,  a  baseline  was  first  established.  This  amounted  to 
a  period  of  eight  months  for  MS&D  and  nine  months  for  Comm/Nav  where  indi¬ 
cator  data  were  collected,  but  no  feedback  was  given  to  the  units.  These  data 
provided  a  baseline  against  which  the  effects  of  the  interventions  could  be  evalu¬ 
ated. 


The  schedule  was  as  follows:  First,  each  of  the  five  units  developed  its 
productivity  measurement  system.  This  was  followed  by  the  baseline  period. 
Next,  feedback  was  given  to  each  unit  for  five  months.  Goal  setting  was  then 
added  to  feedback  for  each  unit,  and  continued  for  another  five  months.  Finally, 
incentives  were  added  to  feedback  and  goal  setting  in  each  unit  for  five  months. 

Feedback 

Productivity  feedback  consisted  of  formal  computer-generated  reports  that 
were  given  to  the  personnel  in  each  section  monthly.  To  develop  these  reports, 
we  first  produced  examples  of  what  the  basic  report  would  look  like  and  asked 
for  supervisors’  thoughts  on  how  best  to  present  the  material  for  maximum  clar¬ 
ity.  We  also  proposed  some  other  information  that  they  might  find  useful,  and 
asked  for  their  suggestions  on  items  to  be  included.  After  several  revisions,  a 
final  version  of  the  productivity  feedback  report  was  developed.  An  example  of 
the  monthly  report  for  the  Comm/Nav  shop  is  presented  in  Figure  6. 

The  first  page  of  the  report  provides  the  basic  productivity  data.  It 
shows  the  products  and  indicators,  the  indicator  data  for  that  month,  and  the  ef¬ 
fectiveness  score  associated  with  that  level  of  each  indicator.  The  lower  portion 
of  the  page  shows  the  total  effectiveness  for  each  of  the  products  and  finally, 
the  overall  effectiveness  for  the  shop.  The  second  page  of  the  report  adds  in¬ 
formation  to  the  basic  data.  The  top  half  of  the  report  shows  the  change  in 
productivity  from  the  previous  month  to  the  current  month.  The  indicator  data 
and  effectiveness  scores  for  both  the  previous  month  and  the  current  month  are 
shown,  as  are  the  changes  in  effectiveness  from  last  month  to  the  current  month. 
This  part  of  the  report  was  requested  by  shop  personnel  to  aid  them  in  di¬ 
agnosing  areas  where  they  were  increasing  or  decreasing  in  productivity. 
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Sample  Feedback  Report 


Figure  6. 


PRODUCTIVITY  REPORT 
COMM/NAV  SHOP 
CRS  MAINTENANCE 


INDICATOR  AND  EFFECTIVENESS  DATA  FOR  MARCH 


INDICATOR  EFFECTIVENESS 


INDICATOR 

DATA 

SCORE 

EQUIPMENT  REPAIR 

BOUNCES 

3.1 

76 

%  QA  INSPECTIONS  PASSED 

90.9 

30 

AWM 

13.5 

80 

AWP 

39.6 

29 

DEMAND  MET 

91.7 

63 

TRAINING 

STS  TASKS  COMPLETED 

8 

35 

%  QUAL  TASKS  COMPLETED:  COM 

69.5 

72 

%  QUAL  TASKS  COMPLETED:  NAV 

56.8 

68 

SCHED  TRAINING  TASKS  OVERDUE 

0 

10 

OTHER  DUTIES 

MOBILITY  EQUIPMENT  OVERDUE 

0 

25 

PMEL  OVERDUE 

0 

25 

%  349  ERRORS 

1 

40 

MISSED  APPOINTMENTS 

0 

10 

EFFECTIVE*  ESS 


TOTALS  SCORE 

EQUIPMENT  REPAIR  278 

TRAINING  185 

OTHER  DUTIES  100 

OVERALL  EFFECTIVENESS  563 


r 


Figure  6.  (Concluded) 


HPPBCTifBWBSS  CHAWQB  PKOM  F 


UJJtT  TO  MAKC* 


INDICATOR 

DATA: 

EPP. 

INDICATOR 

DATA: 

EPF. 

CHANGE 

PERRUARY 

SCORE 

MARCH 

SCORE 

IN  EPP. 

BOUNCES 

2.8 

81 

3.1 

76 

-5 

%  QA  INSPECTIONS 

91.7 

34 

90.9 

30 

-4 

AWM 

15.58 

72 

13.5 

80 

8 

AWP 

40.8 

27 

39.6 

29 

2 

DEMAND  MET 

91.5 

59 

91.7 

63 

4 

STS  TASKS  COMPLETED 

9 

52 

8 

35 

-17 

EQUAL  TASKS-COMM 

68.6 

72 

69.5 

72 

0 

EQUAL  TASKS-NAV 

59.5 

71 

56.8 

68 

-3 

SCHEDULED  TRAINING 
TASKS  OVERDUE 

0 

10 

0 

10 

0 

MOBILITY  EQUIPMENT 

OVERDUE 

0 

25 

0 

25 

0 

PMEL  OVERDUE 

0 

25 

0 

25 

0 

E  349  ERRORS 

2 

27 

1 

40 

13 

MISSED  APPOINTMENTS 

0 

10 

0 

10 

0 

CHANGE  TOTALS 

EQUIPMENT  REPAIR 

5 

TRAINING 

-20 

OTHER  DUTIES 

13 

OVERALL  EFFECTIVENESS 

-2 

POTENTIAL  EFFECTIVENESS  GAINS  FOR  NEXT  MONTH 


FROM 

TO 

GAIN 

BOUNCES 

3.1 

0.4 

17 

QA  INSPECTIONS 

90.9 

100 

45 

AWM 

13.5 

0 

15 

AWP 

39.6 

22.6 

48 

DEMAND  MET 

91.7 

95.2 

37 

STS  TASKS  COMPLETED 

8 

9 

17 

EQUAL  TASKS  COMP:  COMM 

69.5 

76 

0 

EQUAL  TASKS  COMP:  NAV 

56.8 

62.8 

4 

SCHED  TUNING  TSKS  OVERDUE 

0 

0 

0 

MOBILITY  EQ  OVERDUE 

0 

0 

0 

PMEL  OVERDUE 

0 

0 

0 

349  ERRORS 

1 

0 

15 

MISSED  APPOINTMENTS 

0 

0 

0 

37 
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The  bottom  half  of  the  page  it  the  information  on  priorities  for  increasing 
productivity.  For  each  indicator  there  it  a  cohnaa  labeled  FROM,  TO,  and 
GAIN.  The  FROM  column  it  the  amount  of  the  indicator  done  for  the  current 
month.  The  TO  column  is  the  amount  of  die  indicator  that  represents  an  in¬ 
crease  of  ooe  unit  on  the  contingency,  and  the  GAIN  cohlnm  indicates  die  gain  in 
effectiveness  that  would  be  achieved  by  such  an  increase.  For  example,  for  re¬ 
pair  demand  met,  if  the  shop  went  from  their  March  level  of  91.7%  to  95.2%  in 
April,  effectiveness  would  increase  by  37  points.  Examination  of  the  GAIN  col¬ 
umn  indicates  that  for  the  next  month,  the  shop  would  increase  their  productivity 
the  most  by  focusing  on  Quality  Assurance  Inspections,  and  the  number  of  units 
Awaiting  Parts  (AWP).  These  show  potential  gains  in  effectiveness  of  45  and 
48  respectively.  It  would  not  be  useful  to  devote  attention  to  training  in  Comm 
Qualification  Tasks,  trying  to  further  decrease  Overdue  Scheduled  T  raining 
Tasks,  or  any  of  the  other  indicators  that  have  a  gain  value  of  zero  or  near 
zero.  This  information  can  therefore  serve  as  a  basis  for  determining  priorities 
for  the  next  month.  It  suggests  those  areas  where  the  unit  should  focus  to  pro¬ 
duce  maximum  gains  in  effectiveness.  The  feedback  report  for  MS&D  was 
similar  to  this  Comm/Nav  report.  A  copy  of  this  report  is  included  in  Pritchard, 
et  al.  (1967a). 

The  calculation  of  the  GAIN  amount  is  based  on  the  amount  of  increase  in 
effectiveness  that  would  occur  with  an  increase  of  "one  unit”  of  the  indicator. 
The  size  of  a  one  unit  increase  was  determined  from  the  indicator  values  used 
in  the  contingencies.  If  the  indicator  values  in  a  contingency  were  2%,  4%,  6%, 
8%,  etc.  the  size  of  a  one  unit  increase  for  that  indicator  was  2%.  If  the  indi¬ 
cator  values  were  50,  60,  70,  80,  etc.,  the  size  of  a  one  unit  increase  was  10. 
The  contingencies  were  originally  developed  so  that  the  number  of  increments  for 
the  different  contingencies  was  as  equal  as  possible  so  that  a  "one  unit"  incre¬ 
ment  was  roughly  comparable  across  the  different  contingencies.  Once  the  size 
of  the  "one  unit"  increase  was  determined  for  each  contingency,  the  TO  figure 
was  calculated  by  adding  the  one  unit  increase  to  the  actual  value  of  the  indicator 
for  the  preceding  month.  If  the  last  month’s  indicator  level  was  83.6  and  the 
size  of  ooe  unit  was  10,  the  FROM  value  would  be  83.6,  the  TO  value  would  be 
93.6,  and  the  GAIN  value  would  be  determined  by  what  the  contingency  indicated 
as  the  gain  in  effectiveness  if  the  unit  went  from  83.6  to  93.6  on  that  indicator. 

» 

There  was  one  special  circumstance  that  had  to  be  dealt  with  using  this 
approach.  It  was  possible  for  the  TO  value  to  be  higher  than  the  maximum 
value  of  die  indicator.  This  occurred  when  the  unit  was  already  high  on  that  in¬ 
dicator  and  increasing  "one  unit"  would  put  them  over  the  maximum.  It  also  oc¬ 
curred  occasionally  if  the  unit  was  already  over  the  maximum  on  that  indicator. 
This  was  dealt  with  by  using  the  maximum  possible  effectiveness  value  for  the 
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indicator  aa  the  upper  limit  in  effectiveness.  In  other  words,  if  the  effective¬ 
ness  value  for  being  at  the  maximum  of  the  indicator  was  +75,  this  was  the 
maximum  effectiveness  score  that  could  be  gained  from  that  indicator.  If  the 
unit  was  near  the  maximum  already  with,  for  example,  a  past  month’s  indicator 
level  which  yielded  an  effectiveness  score  of  +73,  the  most  they  could  improve 
would  be  to  die  value  of  the  ceiling,  +75,  for  a  maximum  gain  of  only  +2. 

One  feature  of  the  system  is  that  it  allows  one  to  directly  compare  the 
productivity  of  very  different  units.  This  feature  was  very  important  to  the  su¬ 
pervisors  and  managers  of  the  MS&D  branch  since  it  allowed  them  to  compare 
the  four  sections  of  the  branch.  In  order  to  make  this  comparison,  we  first 
determined  the  maximum  possible  overall  effectiveness  for  each  section.  This 
was  done  by  determining  the  effectiveness  value  for  the  maximum  possible  value 
of  each  indicator,  and  summing  these  effectiveness  values.  The  resultant  score 
represented  the  effectiveness  value  that  would  occur  if  the  unit  was  doing  as 
well  as  it  was  possible  to  do  on  every  aspect  of  their  work;  in  other  words, 
their  maximum  possible  overall  effectiveness.  Recall  that  these  maximums  were 
developed  by  consensus  among  the  supervisors  of  the  units,  and  were  discussed 
and  approved  by  management.  Thus,  they  should  represent  realistic  maximums, 
and  the  effectiveness  scores  represent  the  value  of  the  maximum  contribution 
each  of  the  units  could  make  to  the  organization. 

Once  the  maximum  possible  effectiveness  was  calculated,  the  actual 
monthly  overall  effectiveness  score  for  each  section  was  expressed  as  a  percent¬ 
age  of  maximum  possible  effectiveness.  This  percentage  of  maximum  effective¬ 
ness  was  the  measure  by  which  each  unit  was  compared  to  the  other.  These 
data  were  included  in  the  monthly  feedback  report  for  each  section  of  MS&D. 

The  feedback  report  was  generated  each  month  for  15  months  for  each  of 
the  five  units.  It  was  presented  within  three  workdays  after  the  end  of  the 
month,  and  a  copy  was  given  to  each  individual  in  the  chain  of  command,  from 
the  section  supervisors  to  the  Deputy  Commander.  A  copy  was  also  posted  in  the 
working  area  of  each  section  so  incumbents  could  review  it.  In  addition,  graphs 
were  posted  in  the  work  area  and  updated  each  month:  one  for  overall  effec¬ 
tiveness,  and  one  for  each  indicator.  These  graphs  allowed  unit  personnel  to  see 
changes  in  effectiveness  over  time.  As  one  might  imagine,  both  the  feedback  re¬ 
port  and  the  graphs  generated  considerable  interest  when  they  were  posted  each 
month. 


Once  the  feedback  reports  were  circulated,  a  meeting  was  held  with  in¬ 
cumbents,  supervisors  and,  for  MS&D,  a  management  representative  to  review  the 
feedback  report  for  the  month.  Areas  of  improvement  were  noted  and  areas  of 
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decrease  discussed.  Reasons  for  the  improvements  or  decreases  were  consid¬ 
ered,  and  aqy  longer- range  trends  were  noted.  This  meeting  also  served  as  a 
basis  for  planning  priorities  for  die  next  month,  and  for  altering  work  strategies 
to  improve  productivity. 

Goal  Setting 

After  five  months  of  feedback,  goal  setting  was  added  to  the  receiving  of 
feedback  reports.  The  first  step  in  implementing  goal  setting  was  to  train  the 
personnel  in  setting  productivity  goals.  The  nature  and  process  of  goal  setting 
was  explained  to  them,  and  the  importance  of  their  participation  was  stressed. 
They  were  told  the  importance  of  setting  a  difficult  but  attainable  goal.  Prior  to 
the  start  of  the  goal  setting  program,  a  meeting  was  held  with  supervisors  and 
incumbents  to  explain  how  the  program  would  work.  They  were  told  to  discuss 
goals  among  themselves  for  a  few  days.  After  this  time,  die  first  goal  setting 
sessions  were  held.  These  were  attended  by  members  of  the  research  team  to 
facilitate  the  process,  and  especially  to  ensure  participation  and  to  encourage 
them  to  set  reasonable  goals. 

One  issue  that  became  quite  significant  was  whether  the  goals  would  be 
"reportable”  or  "non- reportable."  Unit  personnel  were  reluctant  to  set  goals  that 
represented  difficult  but  attainable  goals,  if  the  goals  were  to  be  reported  to 
higher  management.  They  feared  that  they  might  be  held  to  whatever  goals  they 
set  and  would  thus  receive  a  negative  evaluation  if  these  goals  were  not  met, 
even  if  productivity  had  actually  increased.  It  was  decided  that  public  or  re¬ 
portable  goals  would  be  set  at  a  level  that  would  be  low  enough  to  ensure  that 
the  unit  would  exceed  them.  Such  goals  would  have  little  motivational  force,  and 
fail  to  have  a  positive  effect  on  productivity.  Thus,  the  goal  setting  system  was 
designed  to  be  non- reportable;  that  is,  the  goals  would  be  for  section  use  only, 
and  would  not  be  communicated  to  higher  management.  Section  incumbents  and 
supervisors  set  the  goals,  members  of  the  unit  knew  what  the  goals  were,  but 
the  goals  were  not  communicated  beyond  branch  supervisors.  Upper  management 
agreed  to  the  use  of  non- reportable  goals.  They  felt  that  if  productivity  contin¬ 
ued  to  be  as  high  as  it  had  been  during  feedback,  or  if  it  improved,  they  did  not 
need  to  know  the  unit  goals  that  helped  produce  such  results. 

Comm/Nav  and  each  of  the  four  units  of  MS&D  set  goals  for  themselves 
in  terms  of  their  overall  effectiveness  for  the  following  month.  Goals  were  set 
jointly  by  incumbents  and  supervisors.  When  the  month-end  feedback  report  was 
produced,  the  unit  personnel  noted  whether  or  not  they  had  attained  the  goal,  and 
discussed  reasons  why  die  goal  was  or  was  not  met.  The  goal  was  then  re¬ 
viewed  and  reset  for  the  following  month. 
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Incentive* 


After  five  months  of  feedback  plus  goal  setting,  incentives  were  added  as 
the  final  treatment.  Following  considerable  discussion  and  input  front  samples  of 
incumbents  up  to  senior  management,  the  incentive  chosen  was  time-off  from 
work.  Specifically,  unit  personnel  were  given  a  half  day  or  full  day  off  if  the 
unit  achieved  die  level  of  productivity  needed  to  qualify  for  the  incentive.  This 
incentive  seemed  powerful  for  the  personnel  involved  and  was  feasible  to  imple¬ 
ment. 


Several  other  issues  had  to  be  dealt  with  in  designing  die  incentive  system. 
One  was  the  number  of  levels  of  productivity  diet  would  be  defined  as  resulting 
in  different  incentive  levels.  At  one  extreme  is  a  system  where  there  is  a  sin¬ 
gle  level  of  productivity  that,  if  reached,  leads  to  an  incentive.  At  the  other  ex¬ 
treme  is  a  piece-rate-type  system  where  each  increase  in  productivity  leads  to  an 
increase  in  the  amount  of  incentive.  The  advantages  of  the  single-level  approach 
are  that  it  is  simple  to  administer  and  can  be  used  with  any  type  of  incentive. 
The  advantage  of  the  multiple-level  approach  is  that  no  matter  where  the  unit's 
productivity  is,  there  is  still  an  incentive  to  increase  productivity  further.  With 
the  single-level  approach,  once  the  level  of  productivity  that  produces  the  incen¬ 
tive  is  reached,  there  is  no  incentive  to  increase  productivity  further. 

Another  design  issue  concerned  the  effects  of  asking  the  units  for  higher 
and  higher  levels  of  productivity.  At  the  time  the  incentive  system  was  being  de¬ 
signed,  all  the  units  had  greatly  increased  their  productivity.  In  one  sense,  the 
system  had  asked  them  to  increase  their  productivity  during  feedback  and  again 
during  goal  setting.  If  the  incentive  system  asked  for  even  greater  improvement 
before  any  incentives  were  awarded,  we  felt  there  was  a  possibility  of  a  nega¬ 
tive  reaction  to  the  program. 

There  was  also  an  issue  of  setting  the  incentive  levels  so  that  they  were 
equitable  across  units.  For  Cotnm/Nav  this  was  not  a  problem  since  they  had 
little  contact  with  MS&D.  However,  the  four  sections  of  MSAD  were  in  constant 
contact  with  one  another,  and  the  system  had  to  be  designed  such  that  no  one 
section  had  an  easier  time  obtaining  time-off  than  another.  If  the  participants 
judged  the  system  to  be  inequitable,  it  could  not  be  effective. 

Finally,  there  was  the  issue  of  whether  to  use  branch-level  incentives  in 
addition  to  the  section-level  incentives  in  MSAD.  The  four  units  of  MSAD  had  to 
cooperate  extensively  to  get  the  work  done.  It  would  have  been  possible  for  a 
given  section  to  do  things  that  maximised  their  overall  effectiveness  but  which 


caused  problems  for  the  other  sections.  Therefore,  the  incentive  system  had  to 
have  a  mechanism  for  fostering  cooperation  among  the  units. 

The  incentive  system  was  designed  with  these  issues  in  mind.  The  sys¬ 
tem  used  two  levels  of  the  incentive,  a  half  day  off  and  a  full  day  off. 

Comm/Nav  received  a  half  day  off  if  their  overall  effectiveness  exceeded 
the  mean  overall  effectiveness  for  the  five  immediately  preceding  months.  If 
they  exceeded  this  level  by  a  meaningful  amount  (as  determined  jointly  by  incum¬ 
bents,  supervision,  and  management),  they  received  a  full  day  off.  It  was  de¬ 
cided  that  for  the  full  day  off  to  be  awarded,  overall  effectiveness  must  exceed 
the  mean  of  the  last  five  months  by  5%.  Since  Comm/Nav  was  very  close  to 
their  maximum  possible  effectiveness  by  this  time,  a  5%  increase  was  judged  to 
be  a  substantial  improvement.  One  problem  with  this  system  was  that  if  pro¬ 
ductivity  continued  to  increase,  the  mean  of  the  last  five  months  would  continue 
to  rise.  This  would  mean  that  it  would  be  more  and  more  difficult  for  the  unit 
to  achieve  the  productivity  level  necessary  for  incentives.  Eventually,  continued 
increases  in  productivity  would  make  it  impossible  to  get  any  incentives.  To 
deal  with  this  problem,  a  maximum  was  set  on  the  overall  effectiveness  level 
necessary  to  get  the  incentives.  For  the  half  day  off,  the  maximum  level  was 
set  at  85%  of  their  highest  possible  overall  effectiveness;  for  the  full  day  off, 
the  maximum  was  set  at  94%. 

MS&D  also  had  two  levels  of  the  incentive,  but  with  a  different  structure. 
A  given  section  received  a  half  day  off  if  its  overall  effectiveness  exceeded  the 
mean  overall  effectiveness  of  the  five  highest  productivity  months  prior  to  the 
start  of  incentives.  In  practice,  this  meant  that  they  would  receive  the  incentive 
if  their  productivity  exceeded  the  mean  productivity  under  goal  setting.  There 
was  also  a  branch-level  incentive  of  an  additional  half  day  off  to  be  given  to 
each  section  if  the  branch  reached  its  incentive  level  on  overall  effectiveness. 
This  productivity  level  for  the  branch  to  get  the  additional  half  day  off  was  es¬ 
sentially  the  sum  of  the  section-level  goals.  That  is,  the  individual  section  in¬ 
centives  were  given  when  the  section’s  productivity  continued  at  a  very  high  level. 
To  get  the  branch  incentive,  productivity  had  to  be  maintained  at  this  high  level 
for  each  section.  This  was  done  since  it  was  felt  that  several  of  the  sections 
were  already  performing  very  near  the  maximum  possible  level,  and  asking  them 
to  go  above  this  to  get  the  full  day  off  as  in  Comm/Nav  was  unreasonable.  If  a 
section  reached  its  incentive  level  and  the  branch  reached  its  level,  that  section 
received  a  full  day  off.  If  the  section  reached  its  level  but  the  branch  did  not, 
the  section  received  a  half  day  off.  If  the  section  did  not  meet  its  incentive 
level  but  the  branch  did,  the  section  received  a  half  day  off. 


Thus,  the  system  dealt  with  the  issue  of  asking  for  higher  and  higher 
productivity  in  Comm/Nav  by  having  one  level  of  the  incentive  for  continuing  to 
maintain  previous  high  levels  of  productivity,  and  a  second  level  of  incentive  for 
exceeding  that  productivity.  For  MS&D,  this  issue  was  dealt  with  by  requiring 
that  a  section  maintain  its  high  level  of  productivity  for  the  half  day  off,  and 
that  all  sections  maintain  high  productivity  for  the  full  day  off.  A  branch-level 
incentive  was  used  in  MS&D  in  addition  to  the  section-level  incentive.  This  in¬ 
centive  was  structured  in  such  a  way  that  sections  were  encouraged  to  cooperate. 
To  get  the  greater  incentive,  the  entire  branch  had  to  do  well.  A  section  could 
have  a  bad  month  and  still  get  a  half  day  off  if  they  could  help  the  rest  of  the 
sections  pull  up  the  branch  total.  Another  possibility  was  that  a  section  that  was 
doing  extremely  well  would  help  another  section  so  that  both  would  get  time  off. 

The  issue  of  equity  across  sections  in  MS&D  was  dealt  with  by  first  ex¬ 
plaining  the  issue  of  equity  to  the  personnel,  and  then  having  incumbents,  supervi¬ 
sors,  and  management  of  the  different  sections  jointly  determine  what  levels  of 
productivity  should  be  reached  to  obtain  incentives  in  each  of  the  sections. 

During  the  incentive  treatment,  the  units  continued  to  set  their  own  per¬ 
formance  goals  and  review  them  each  month.  We  felt  that  perhaps  after  the  in¬ 
centive  treatment  started  they  might  simply  set  their  goals  equal  to  the  level 
needed  to  receive  time-off,  and  thus,  continuing  goal  setting  might  be  unneces¬ 
sary.  However,  the  goal  setting  was  continued  so  that  the  individual  units  could 
set  different  goals  if  they  so  desired,  and  so  that  the  integrity  of  the  experi¬ 
mental  design  was  maintained.  Specifically,  by  allowing  units  to  continue  their 
own  goal  setting,  we  could  assess  the  incremental  effects  of  incentives  when 
added  to  goal  setting.  If  goal  setting  had  been  discontinued,  assessment  of  this 
incremental  effect  would  not  have  been  possible. 

IV.  RESULTS 


The  results  of  the  effort  will  be  presented  in  five  sections:  (a)  de¬ 
velopment  of  the  productivity  measurement  system;  (b)  effects  on  productivity; 
(c)  results  for  the  feedback,  goal  setting,  and  incentive  interventions;  (d)  attitude 
data;  and  (e)  effects  after  the  departure  of  the  research  team. 

» 

Development  of  the  Productivity  Measurement  System 

There  were  a  number  of  results— both  qualitative  and  quantitative-perti¬ 
nent  to  the  development  of  the  productivity  measurement  system. 
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One  qualitative  result  was  the  change  in  the  attitudes  of  the  unit  personnel 
during  the  development  of  the  system.  When  we  first  started  working  with 
them,  their  attitudes  toward  the  project  were  mixed.  Although  some  unit  person¬ 
nel  were  positive,  others  were  more  skeptical.  Those  with  the  more  negative 
attitudes  felt  that  the  researchers  were  non-military  outsiders  who  did  not  know 
the  details  of  their  work  situation  but  were  going  to  impose  a  management  sys¬ 
tem  about  which  these  personnel  had  reservations.  Though  these  people  were 
willing  to  cooperate,  they  definitely  had  their  doubts.  Through  the  course  of  the 
system  development,  their  attitudes  changed  dramatically.  By  the  time  system 
development  was  completed,  most  unit  personnel  h H  positive  attitudes  toward  the 
effort.  They  were  solidly  behind  the  system,  felt  positive  toward  the  re¬ 
searchers,  and  were  quite  disappointed  that  they  had  to  wait  through  the  neces¬ 
sary  months  of  baseline  before  feedback  could  begin. 

A  second  qualitative  finding  deals  with  interpreting  the  strength  of  the  pro¬ 
ductivity  results.  Development  of  the  system,  by  its  very  nature,  led  the  partici¬ 
pants  to  examine  unit  objectives,  suggest  possible  measures  of  these  objectives, 
and  evaluate  their  productivity  expectations  and  limits.  This  process  led  unit  su¬ 
pervisors  to  see  numerous  places  where  improvements  could  be  made  in  the  op¬ 
eration  of  the  units.  They  began  to  implement  these  changes.  This  created  a 
real  dilemma  for  the  researchers.  Although  it  was  certainly  worthwhile  for  the 
units  to  improve  their  effectiveness  because  of  the  development  of  the  system, 
this  improvement  was  occurring  prior  to  the  start  of  our  baseline.  If,  because 
of  this,  the  baseline  period  showed  higher  effectiveness  than  it  otherwise  would 
have,  there  would  be  a  decrease  in  the  size  of  any  feedback  effect.  There  was 
little  the  researchers  could  do  about  this  dilemma.  The  units  felt  strongly  that 
such  changes  should  be  made,  and  made  them.  They  felt  these  changes  were 
increasing  their  effectiveness,  and  this  indeed  seemed  to  be  the  case,  based  on 
what  little  data  were  available  at  the  time.  Interviews  with  supervisors  indicated 
that  they  believed  a  substantial  portion  of  this  improvement  was  due  to  the  pro¬ 
cess  of  developing  the  productivity  measurement  system.  This  suggests  that  the 
improvements  in  productivity  that  were  evidenced  in  the  interventions  were,  in 
fact,  underestimates  of  the  overall  impact  of  the  development  and  introduction  of 
feedback  from  the  productivity  measurement  system. 

The  reliability  and  validity  of  the  system  were  also  assessed  during  devel¬ 
opment  of  the  system.  Reliability  was  assessed  by  examining  interjudge 
agreement  on  the  contingencies.  The  Comm/Nav  shop  had  two  shifts  in 
operation.  Personnel  from  both  shifts  were  involved  with  the  development  of 
products  and  indicators.  To  assess  reliability,  we  developed  two  independent  sets 
of  contingencies;  this  produced  two  effectiveness  scores  for  each  value  of  each 
indicator:  one  set  from  the  day  shift  and  one  set  from  the  night  shift. 
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Correlations  calculated  between  the  two  sets  of  values  for  each  contingency 
ranged  from  a  low  of  .86  to  a  high  of  .99,  with  an  average  of  .95.  Thus,  the 
reliability  of  the  contingencies  as  measured  by  interjudge  agreement  was  quite 
high. 

The  validity  of  the  system  was  evaluated  using  five  different  productivity 
scenarios  of  hypothetical  indicator  data  developed  for  Comm/Nav.  This  was  done 
by  selecting  a  reasonable  value  for  each  indicator  in  such  a  way  that  the  differ¬ 
ent  scenarios  varied  as  to  their  overall  effectiveness.  Although  the  overall  ef¬ 
fectiveness  of  each  of  the  five  scenarios  varied,  the  differences  were  not  so 
large  as  to  be  completely  obvious.  Also,  the  changes  in  indicator  values  varied, 
but  not  always  in  the  same  direction.  That  is,  although  the  overall  effectiveness 
went  up  in  a  given  scenario,  some  indicator  values  went  down,  while  others  went 
up.  Six  Comm/Nav  supervisors  were  then  given  the  indicator  data  on  the  five 
scenarios  and  asked  to  rank  the  scenarios  as  to  their  overall  effectiveness.  If 
the  system  accurately  reflects  relative  importance,  having  supervisors  rate  the 
scenarios  without  knowing  the  scenario  overall  effectiveness  scores  should  pro 
duce  ratings  which  are  highly  correlated  with  overall  effectiveness  as  calculated 
by  the  system. 

These  ratings  were  done  approximately  two  months  after  the  development 
of  the  system  had  been  completed,  but  before  any  productivity  feedback  had 
started.  Results  showed  a  correlation  of  1.0  between  each  supervisor’s  rankings 
and  the  overall  effectiveness  score  calculated  by  the  system.  This  constitutes 
additional  evidence  for  the  validity  of  the  system. 

Finally,  data  were  also  collected  on  subjective  reactions  to  the  productivity 
measurement  and  feedback  system.  All  incumbents  and  supervisors  (N  =  97, 
which  includes  some  supervisors  not  technically  part  of  the  units,  but  in  their 
chain  of  command)  were  surveyed  after  several  months  of  experience  with  the 
feedback  system.  The  survey  asked  for  their  reactions  to  different  aspects  of 
the  feedback  system.  The  survey  items  consisted  of  5-point  Likert  scales  with 
response  formats  ranging  from  Strongly  Agree  to  Strongly  Disagree.  Responses 
to  these  items  were  uniformly  positive.  Table  1  presents  the  items  and  the  per¬ 
centage  of  respondents  who  agreed  or  strongly  agreed  and  the  percentage  who 
disagreed  or  strongly  disagreed.  After  reverse-scoring  the  negatively  phrased 
item  (#12),  the  mean  percentage  of  respondents  across  all  items  who  Strongly 
Agreed  or  Agreed  was  64.7%,  while  the  corresponding  percentage  who  Disagreed 
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Table  1.  Subjective  Evaluation  of  the  System 


ITEM 

PERCENT 
AGREE  OR 
STRONGLY 
AGREE 

PERCENT 
DISAGREE  OR 
STRONGLY 
DISAGREE 

1.  The  feedback  system  tells  me  how 
good  a  job  1  am  doing. 

64 

4 

2.  The  feedback  system  tells  me  how 
good  a  job  the  section  is  doing. 

87 

1 

3.  The  feedback  system  helps  me  see 
the  section’s  priorities. 

77 

6 

4.  The  feedback  system  helps  the 
section  be  more  productive. 

61 

4 

5.  A  system  like  this  would  help  other 
Air  Force  bases  be  more  productive. 

62 

4 

6.  The  feedback  system  is  clear  and 
understandable. 

58 

7 

7.  It  was  worth  the  effort  to  develop 
the  feedback  system. 

64 

10 

8.  It  was  worth  the  effort  to  keep  the 
feedback  system  in  operation. 

62 

10 

9.  The  information  about  section 
performance  that  goes  into  the  feedback 
system  is  accurate. 

52 

13 

10.  The  feedback  system  gives  a  good 
measure  of  productivity. 

64 

13 

11.  Overall,  I  like  the  .cedback  system. 

62 

9 

12.  I  would  prefer  not  to  have  this 
feedback  system  at  tEe  next  organization 

I  work  in. 

13 

54 

13.  The  feedback  system  is  a  better 
way  of  measuring  productivity  than  what 
the  section  used  to  have. 

75 

0 

MEANS  « 

64.7 

7.2 

MEANS 


or  Strongly  Disagreed  was  7.2%.  Clearly,  the  response  to  the  system  by  those 
who  used  it  was  very  positive. 

Effects  On  Productivity 


Once  the  system  was  developed  and  a  baseline  was  established,  the  system 
was  used  to  generate  feedback.  Next,  goal  setting  was  added  to  feedback,  and 
finally  incentives  were  added. 

The  overall  effects  across  the  five  units  are  shown  in  Figure  7,  which 
plots  the  mean  of  the  overall  effectiveness  of  the  five  sections  over  time.  As 
the  figure  indicates,  overall  effectiveness  increased  substantially  over  the  base¬ 
line. 


In  order  to  calculate  a  single  index  of  change  in  productivity,  we  calculated 
the  percent  gain  from  the  baseline  compared  to  maximum  possible  gain.  That  is, 
mean  overall  effectiveness  was  calculated  from  the  baseline.  The  difference 
between  this  value  and  the  maximum  possible  overall  effectiveness  the  section 
could  obtain  was  calculated.  This  maximum  possible  effectiveness  was  deter¬ 
mined  by  calculating  the  overall  effectiveness  score  if  the  unit  produced  at  the 
maximum  possible  value  for  each  of  the  indicators.  The  gain  in  overall  effec¬ 
tiveness  from  baseline  to  each  treatment  was  then  calculated,  and  expressed  as 
the  percentage  of  maximum  possible  gain.  For  example,  assume  that  the  baseline 
mean  was  400  and  the  maximum  possible  overall  effectiveness  was  800.  If  the 
mean  overall  effectiveness  during  feedback  was  600,  this  would  be  a  gain  of 
200.  This  is  50%  of  the  maximum  possible  gain,  and  would  be  the  value  re¬ 
ported.  This  approach  to  calculating  change  is  in  some  ways  conservative,  in  the 
sense  that  the  maximum  possible  increase  is  limited  to  100%. 

Using  this  approach,  the  average  increase  over  baseline  was  50%  for  feed¬ 
back,  75%  for  goal  setting,  and  76%  for  incentives.  Figure  8  presents  similar 
data  for  Comm/Nav  alone.  The  data  show  average  increases  of  30%  for  feed¬ 
back,  65%  for  goal  setting,  and  68%  for  incentives.  Figure  9  presents  the  data 
for  MS&D  alone.  Average  percent  increases  were  54%,  77%,  and  79%.  Finally, 
Figures  10,  11,  12,  and  13  show  the  productivity  effects  for  each  of  the  four 
sections  of  MS&D. 

These  results  indicate  a  major  increase  in  productivity.  The  effects  were 
extremely  large.  In  addition,  the  MS&D  effects  were  consistent  across  sections. 
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FIG.  7  PRODUCTIVITY 
OF  ALL  FIVE 
SECTIONS  COMBINED 
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FIG.  12  PICKUP  &  DELIVERY 
OVERALL  EFFECTIVENESS 


FIG.  13  INSPECTION 
OVERALL  EFFECTIVENESS 


▼ 


Potential  Problems  of  Interpretation 

Before  we  can  confidently  attribute  these  effects  to  the  experimental  in¬ 
terventions,  several  issues  must  be  considered.  The  first  is  the  possible  pres¬ 
ence  of  a  Hawthorne  effect,  where  productivity  could  have  increased  simply  be¬ 
cause  the  units  were  singled  out  for  the  special  treatment  of  being  in  a  research 
project.  Although  such  an  effect  is  indeed  possible,  the  project  was  designed  to 
avoid  possible  contamination  from  such  an  effect.  Specifically,  the  initial  contact 
with  the  units  during  system  development  was  quite  intensive,  and  if  a  Hawthorne 
effect  was  going  to  occur,  it  should  have  occurred  then.  Since  this  was  well 
before  the  start  of  the  baseline,  any  productivity  increase  would  have  already  oc¬ 
curred  and  would  not  contaminate  the  results.  Thus,  the  presence  of  a 
Hawthorne  effect  cannot  explain  the  productivity  increases. 

Another  possibility  was  that  the  increases  in  productivity  which  occurred 
during  the  treatments  might  have  been  due  to  increases  in  the  number  of  person¬ 
nel  in  the  units.  For  Comm/Nav,  we  looked  at  the  total  number  of  personnel  in 
the  shop  during  each  month  of  the  effort.  The  mean  number  of  personnel  by 
month  is  presented  in  Figure  14.  The  mean  number  of  personnel  during  baseline 
was  30.9.  This  figure  increased  slightly  during  feedback  to  33.0,  was  32.8  dur¬ 
ing  goal  setting,  and  dropped  to  31.0  during  incentives.  Since  personnel  levels 
during  the  period  of  highest  gain  in  productivity  were  essentially  equal  to  the 
level  during  baseline,  we  concluded  that  the  increases  in  productivity  were  not 
caused  by  an  increase  in  number  of  personnel.  For  MS&D,  the  data  reviewed 
were  total  number  of  personnel,  and  the  number  of  hours  of  overtime  logged  per 
month.  Unlike  Comm/Nav,  MS&D  routinely  had  considerable  overtime.  The 
number  of  personnel  for  MS&D  by  month  is  also  presented  in  Figure  14.  The 
mean  was  51.8  for  baseline,  53.7  for  feedback,  48.4  for  goal  setting,  and  49.2 
for  incentives.  Thus,  the  overall  number  of  personnel  decreased  over  the  period 
of  the  treatments.  The  overtime  data  presented  in  Figure  15  show  that  number 
of  hours  of  overtime  decreased  during  the  time  that  productivity  increased. 
Overtime  went  from  a  mean  of  1,348  hours  per  month  during  baseline  to  892 
during  feedback,  404  during  goal  setting,  and  416  during  incentives.  Thus,  by  the 
end  of  the  treatments,  overtime  was  less  than  one-third  of  what  it  was  during 
baseline.  These  data  indicate  that  by  the  end  of  project,  the  productivity  gains 
that  had  occurred  were  made  with  no  increase  in  number  of  personnel  in 
Comm/Nav,  and  a  decrease  in  manpower  in  MS&D. 

Finally,  it  was  possible  that  there  could  have  been  changes  occurring  in  the 
larger  organizations,  of  which  die  five  experimental  units  were  part,  which 
caused  general  increases  in  productivity  for  all  units.  To  explore  this  possibility 
productivity  data  were  collected  on  several  sections  that  were  similar  to  the 
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FIG.  14  PERSONNEL  FOR 
COMM/NAV  AND  MS&D 
BY  MONTH 


'GOAL  SETTING  l-INCENTIVES 


FIG.  15  MONTHLY  HOURS 
OVERTIME-MS&D 


experimental  groups  in  the  type  of  work  they  did.  The  idea  was  to  compare 
changes  in  the  productivity  of  these  sections  with  changes  in  the  five  experimen¬ 
tal  units.  If  the  productivity  of  these  control  units  increased  as  much  as  that  of 
the  experimental  units,  then  the  experimental  units’  increased  productivity  was 
most  likely  not  attributable  to  the  interventions. 

Control  group  data  for  Comm/Nav  consisted  of  ten  measures  of  productiv¬ 
ity  from  eight  maintenance  units  in  the  Component  Repair  Squadron,  of  which 
Comm/Nav  was  (me  section.  The  data  were  collected  from  baseline  and  put  into 
a  composite  measure  to  express  overall  productivity  of  the  control  groups.  This 
composite  measure  was  the  sum  of  the  ten  measures.  A  plot  of  this  composite 
is  presented  in  Figure  16.  In  Figure  16,  the  letter  B,  F,  G,  or  I  is  used  to  indi¬ 
cate  which  months  of  control  group  data  correspond  to  the  treatments.  The  con¬ 
trol  groups  did  not  receive  these  interventions. 

The  mean  value  across  the  ten  Comm/Nav  control  group  measures  during 
baseline  was  317.  It  dropped  to  295  during  feedback,  and  rose  to  377  and  365 
during  goal  setting  and  incentives,  respectively.  These  results  show  that  the 
control  groups  decreased  somewhat  in  productivity  during  the  Comm/Nav  feedback 
period,  and  increased  thereafter.  We  believe,  therefore,  that  the  productivity  in¬ 
crease  during  feedback  in  Comm/Nav  was  not  due  to  wider  organizational 
changes,  since  the  productivity  of  other  squadron  units  actually  decreased  during 
this  period.  Furthermore,  the  increases  during  the  Comm/Nav  goal  setting  and 
incentives  periods  were  not  present  across  all  units.  These  increases  were 
brought  about  primarily  by  large  increases  in  productivity  on  two  of  the  ten  con¬ 
trol  measures. 

Productivity  data  on  four  control  measures  were  examined  for  MS&D. 
These  measures  reflected  overall  functioning  of  the  Supply  Squadron  (exclusive 
of  MS&D)  and  one  other  unit  outside  of  Supply.  None  of  these  units  was  part  of 
the  interventions.  The  composite  index  is  also  presented  in  Figure  16.  The 
mean  of  these  four  measures  during  baseline  was  516;  it  was  512  for  feedback, 
511  for  goal  setting,  and  518  for  incentives.  Thus,  there  were  essentially  no 
changes  in  productivity  for  the  MS&D  control  groups. 

In  summary,  the  control  group  data  indicate  that  the  effects  on  productivity 
that  occurred  in  the  experimental  units  cannot  be  explained  due  to  wider  organi¬ 
zational  changes  in  productivity. 

Having  examined  the  possibility  of  Hawthorne  effects,  effects  due  to  in¬ 
creases  in  the  number  of  personnel,  and  effects  caused  by  wider  organizational 
changes,  we  believe  it  is  safe  to  conclude  that  the  changes  in  productivity  that 
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occurred  were  attributable  to  the  experimental  interventions.  In  addition,  our 
subjective  impressions  and  the  reactions  of  the  personnel  in  the  units  completely 
convinced  us  that  this  was  the  case. 

Indicator  Data 


We  have  shown  the  positive  effects  on  overall  effectiveness.  Let  us  now 
examine  the  effects  on  the  indicators  themselves.  Tables  2  and  3  present  the 
mean  values  for  each  indicator,  by  treatment,  for  Comm/Nav  and  MS&D,  respec¬ 
tively.  For  example,  Table  2  shows  that,  on  the  average,  there  were  5.9% 
bounces  (i.e.,  5.9%  of  the  items  repaired  were  returned  for  reaccomplishment  of 
repair)  during  the  baseline  period.  During  the  incentives  period,  the  average  de¬ 
creased  to  2.5%. 

In  addition,  the  data  for  the  most  important  indicators  for  each  of  the  five 
units  are  plotted  by  month  in  Figures  17  through  22.  These  were  the  indicators 
judged  most  important  by  unit  personnel,  supervisors,  and  managers.  For 
Comm/Nav  the  two  most  important  indicators  were  percent  of  repair  demand  met 
and  percent  of  bounces.  For  MS&D,  the  most  important  indicators  for  the  Re¬ 
ceiving,  Storage  and  Issue,  and  Pickup  and  Delivery  sections  were  the  speed  with 
which  they  handled  "priority  2"  material.  Priority  2  items  were  the  most  im¬ 
portant  items  they  dealt  with,  usually  a  part  for  an  aircraft  that  could  not  per¬ 
form  its  mission  without  that  part.  The  most  important  indicator  for  Inspection 
was  the  number  of  aircraft  parts  that  had  not  been  inspected  by  the  end  of  the 
day.  The  fewer  the  number  of  parts,  the  more  effective  the  unit. 

These  results  indicate  that  the  effects  of  the  system  were  very  powerful. 
Inspection  of  the  table  indicates  that  major  productivity  increases  occurred.  The 
graphs  show  how  dramatic  some  of  the  changes  were. 

Feedback.  Goal  Setting,  and  Incentive  Results 

We  have  presented  results  on  the  effects  of  feedback,  goal  setting,  and 
incentives;  however,  it  is  instructive  to  focus  on  them  more  directly.  Direct 
comparisons  of  the  three  treatments  are  presented  in  Figures  23  and  24.  Figure 
23  is  a  plot  of  mean  overall  effectiveness  by  treatment.  This  figure  presents 
two  sets  of  results:  overall  effectiveness  summed  across  all  five  sections 
(Comm/Nav  and  MS&D),  and  overall  effectiveness  across  the  four  MS&D  sec¬ 
tions  alone.  Figure  24  presents  similar  results  separately  for  each  of  the  five 
sections  (Comm/Nav  and  MS&D). 
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Table  2.  Comm/Nav  Indicator  Means  by 

Treatment 


INDICATOR 

Baseline 

Feedback 

Goals 

Incentives 

%  Bounces  -  1 

5.9 

4.5 

2.7 

2.5 

%  QA  Inspections  Passed  + 

91.9 

88.7 

90.0 

92.6 

ft  Units  Awaiting  Maintenance  - 

23.3 

18.8 

11.8 

10.4 

ft  Units  Awaiting  Parts  - 

44.6 

39.0 

32.8 

36.9 

%  Demand  Met  + 

88.5 

90.5 

93.9 

92.6 

ft  STS  Tasks  Completed  + 

5.1 

6.4 

6.8 

8.6 

%  Qual.  Tasks,  Comm  + 

57.9 

58.9 

68.1 

69.4 

%  Qual.  Tasks,  Nav  + 

40.0 

42.5 

55.0 

60.1 

ft  Scheduled  Trn.  Overdue  - 

1.3 

0.0 

0.0 

0.0 

ft  Mobility  Eq.  Overdue  - 

.4 

0.0 

0.0 

0.0 

ft  PMEL  Overdue  - 

.9 

0.0 

0.0 

0.0 

ft  349  Errors  - 

8.8 

1.2 

1.3 

1.7 

ft  Missed  Appointments  - 

.2 

1.0 

.2 

0.0 

1.  A  -  after  an  indicator  signifies 

that  a  smaller  mean  is 

higher  productivity, 

a  +  after  an  indicator  signifies  that  a  larger  mean  is  higher  productivity. 
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Table  3.  MS&D  Indicator  Means  by  Treatment 


INDICATOR 


Baseline  Feedback  Goals  Incentives 


Receiving 

In-checking  Errors/ 100  -  1 
Priority  2  Rec  (Min)  - 
Priority  4  Rec  (Hrs)  - 
tt  Whse  Refusals  in  Receiving  - 
tt  Delinquent  Rejects  - 

Storage  and  Issue 
tt  Inspection  Findings  - 
tt  Whse  Refusals  Wrong  Location 
%  Cleared  Off  R36  + 

Priority  2  Iss  (Min)  - 
Priority  3  Iss  (Min)  - 
Priority  4  Iss  (Hrs)  - 
tt  Repeat  Findings  - 

Pickup  and  Delivery 
tt  Delinquent  Turn-ins  - 
Priority  2  Iss  (Min)  - 
Priority  2  Rec  (Min)  - 
Priority  3  (Min)  - 
Priority  4  (Hrs)  - 
Vehicle  Inspection  Score  + 
tt  Reportable  Accidents  - 
tt  Non-reportable  Accidents  - 


22 

13 

0 

0 

118.6 

20.5 

16.2 

16.3 

23.4 

5.3 

6.5 

4.3 

2.8 

1.8 

0.0 

0.0 

137.6 

27.2 

1.9 

1.2 

15.3 

19.0 

12.2 

9.4 

2.5 

.8 

0.0 

.2 

89.1 

96.5 

98.9 

98.3 

60.8 

13.4 

9.5 

8.9 

117.7 

22.3 

15.8 

20.9 

4.8 

10.5 

2.8 

2.7 

0.0 

0.2 

0.2 

0.0 

44.0 

23.8 

0.0 

0.1 

64.0 

35.3 

27.4 

27.1 

98.3 

42.0 

33.1 

34.2 

76.5 

52.0 

35.9 

35.9 

32.5 

20.3 

19.7 

16.6 

93.6 

94.6 

94.4 

95.5 

0.0 

0.0 

0.0 

0.0 

.3 

.4 

.2 

.0 

1.  A  -  after  an  indicator  signifies  that  a  smaller  mean  is  higher  productivity, 
a  +  after  an  indicator  signifies  that  a  larger  mean  is  higher  productivity. 
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Table  3.  (Concluded) 


MS&D  Indicator  Means  by  Treatment 

INDICATOR  Baseline  Feedback  Goals  Incentives 


Inspection 


ft  Incoming  Items  Left  - 

12.9 

5.4 

4.5 

6.0 

ft  Dated  Items  Left  - 

.3 

.4 

.1 

0.0 

ft  Unidentified  Items  Left  - 

17.6 

9.5 

2.2 

5.0 

ft  Aircraft  Parts  Left  - 

2.2 

.7 

.3 

0.0 

ft  Functional  Checks  Left  - 

.6 

.1 

.1 

.4 

ft  Suspect  Items  Left  - 

.5 

0.0 

.1 

0.0 

ft  Late  Monthly  Inspections  - 

4.3 

.4 

0.0 

.2 

ft  Returns  From  DP  - 

1.0 

0.0 

0.0 

0.0 

%  TCTOs  Checked  + 

100.0 

99.8 

100.0 

100.0 

ft  RODS  From  Other  Bases  - 

6.5 

6.8 

5.6 

7.4 

ft  Off  Base  Shipments  Left  - 

1.8 

.2 

.3 

.1 

Branch  Level  Indicator 

ft  Delinquent  Documents  - 

306.6 

139.2 

60.0 

48.2 
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FIG.  17  COMM/NAV  ” 

PERCENT  DEMAND  MET 


FIG. 18  COMM/NAV 
PERCENT  BOUNCES 


FIG.  19  RECEIVING 
DELIVERY  TIMES  P2’S 


9 


FIG.  20  STORAGE  &  ISSUE 
DELIVERY  TIMES  P2’S 
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FIG.  21  PICKUP  &  DELIVERY 
DELIVERY  TIMES  P2  ISSUES 
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FIG. 23  AVERAGE  EFFECTIVENESS 
BY  TREATMENT 
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It  is  clear  from  Figures  23  and  24  that  a  very  consistent  pattern  resulted. 
There  was  a  major  effect  due  to  feedback,  a  smaller  but  still  large  further  in¬ 
crease  during  goal  setting,  and  either  no  increase  or  a  very  small  additional  in¬ 
crease  during  incentives.  It  is  tempting  to  conclude  that  feedback  increased  pro¬ 
ductivity,  goal  setting  increased  it  considerably  more,  and  incentives  have  little 
additional  effect.  However,  we  do  not  believe  this  is  the  correct  interpretation. 
The  continued  effects  of  feedback  and  ceiling  effects  must  be  considered  in  the 
interpretation  of  the  results.  These  issues  will  be  addressed  in  the  Discussion 
section  of  this  paper. 

It  is  also  instructive  to  examine  what  the  units  did  in  terms  of  goal  setting 
and  incentives.  The  goal  setting  data  over  the  ten  months  of  goal  setting  are 
presented  in  Table  4.  Difficulty  of  the  self-set  goals  varied  considerably  across 
the  five  units.  This  can  be  seen  by  the  first  column  of  the  table,  which  indi¬ 
cates  the  number  of  goals  (out  of  the  ten  possible)  where  the  section's  monthly 
goal  was  set  lower  than  productivity  had  been  for  the  previous  month.  The  val¬ 
ues  range  from  zero  for  the  Receiving  and  Inspection  sections  to  eight  for  the 
Storage  and  Issue  section.  Goal  difficulty  can  also  be  seen  from  the  second  col¬ 
umn  of  the  table,  which  shows  how  each  section's  monthly  goals  compared  with 
their  previous  month’s  goals  generally.  That  is,  if  the  unit  set  monthly  goals 
that  were,  on  the  average,  10%  higher  than  their  last  month’s  productivity,  the 
tabled  value  would  be  110.  As  can  be  seen,  most  of  the  units  set  goals  higher 
than  their  previous  months’  productivity.  The  one  exception  was  the  Storage  and 
Issue  section,  which  generally  set  goals  about  5%  below  their  productivity  for  the 
previous  month. 

This  difference  in  goal  difficulty  is  also  reflected  in  the  third  column  of 
the  table,  which  presents  mean  percentage  of  goal  attained.  If,  on  the  average, 
the  unit’s  productivity  equaled  its  goal,  the  value  would  be  100;  if  productivity  ex¬ 
ceeded  the  goal,  the  value  would  be  above  100.  The  fourth  column  shows  the 
number  of  months  in  which  each  unit  achieved  its  goal.  Both  of  these  columns 
indicate  that  the  Storage  and  Issue  section  fairly  consistently  exceeded  its  goals; 
Comm/Nav  did  so  almost  as  well,  and  the  rest  of  the  units  exceeded  their  goals 
less  frequently. 

The  data  indicate  that  all  the  units  except  Storage  and  Issue  were  gener¬ 
ally  setting  goals  that  actually  represented  increases  in  productivity.  Even  though 
there  were  several  months  when  units  set  goals  below  their  previous  month’s 
productivity,  it  was  expected  that  for  some  months  this  would  be  necessary. 
This  usually  occurred  when  a  unit  expected  some  change  in  workload  or  re¬ 
sources  for  the  upcoming  month.  For  example,  if  Pickup  and  Delivery  had  a 
driver  on  leave  for  the  month,  they  would  lower  the  goal  for  that  month. 
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Table  4.  Goal  Data 


UNIT 

Number  of 
Goals  Set 
Below  Last 
Month’s 
Productivity 

Mean 

Percent  Goal 
Was  of  Last 
Month’s 
Productivity 

Mean 

Percent  of 
Goal 

Attained 

Number 
of  Times 
Goal  Was 

Attained 

Comm/Nav 

2 

100.9 

102.5 

6 

Receiving 

0 

103.2 

97.2 

2 

Storage  &  Issue 

8 

94.7 

106.8 

8 

Pickup  &  Delivery  4 

101.1 

99.7 

3 

Inspection 

0 

102.9 

99.4 

3 

The  results  also  suggest  that  the  goals  that  were  set  were  fairly  difficult 
ones  in  each  unit  except  Storage  and  Issue.  Although  the  percent  increase  over 
the  past  month’s  productivity  may  seem  low,  this  number  includes  those  goals  that 
were  set  below  the  past  month’s  productivity  due  to  workload  or  resource 
changes.  In  addition,  all  the  units  were  producing  at  nearly  their  maximums,  es¬ 
pecially  during  the  last  five  to  eight  months  of  goal  setting.  Thus,  even  a  small 
increase  in  productivity  was  difficult  to  achieve. 

Although  Storage  and  Issue  set  substantially  lower  goals  than  did  the  other 
sections,  this  did  not  seem  to  have  a  negative  effect  on  their  productivity.  Their 
percentage  productivity  increase  over  baseline  during  goal  setting  was  80%,  com¬ 
pared  to  an  increase  of  77%  for  all  four  sections  of  MS&D  combined.  When  in¬ 
centives  were  added  to  goal  setting,  their  increase  was  81%  over  baseline,  while 
all  of  MS&D  showed  an  increase  of  79%. 

The  incentive  treatment  was  instituted  for  five  months.  During  that  time 
the  frequency  of  earning  incentives  varied  considerably  among  the  units. 
Comm/Nav  received  time-off  least  frequently.  They  received  one  half  day  and 
one  full  day  over  the  five  opportunities.  The  Receiving  section  and  the  Storage 
and  Issue  section  each  earned  two  half  days  and  three  full  days  off.  Pickup  and 
Delivery  earned  three  half  days  and  two  full  days.  Inspection  earned  four  full 
days  and  no  half  days.  When  time  off  was  earned  during  a  given  month,  the 
units  took  that  time  off  during  the  following  month.  If  Comm/Nav,  for  example, 
earned  time  off  for  their  February  performance,  they  would  take  it  sometime  in 
March.  This  applied  to  all  units  except  Pickup  and  Delivery.  This  unit  was  un¬ 
dermanned  and  could  not  take  their  time  off  in  the  typical  manner.  Instead  they 
took  the  time  at  a  later  date  when  the  work  load  was  less. 

Another  point  of  interest  relates  to  the  way  goal  setting  was  conducted  un¬ 
der  the  incentive  system.  Recall  that  goal  setting  was  continued  independent  of 
incentives.  Thus,  as  part  of  the  incentive  treatment,  there  was  a  level  of  over¬ 
all  effectiveness  established  for  the  full  and  half  days  off;  at  the  same  time, 
there  was  a  level  of  overall  effectiveness  set  by  each  unit  as  a  goal.  We  won¬ 
dered  whether  the  goal  setting  would  be  superseded  by  the  incentive  system;  that 
is,  whether  the  units  would  merely  set  goals  at  a  level  that  would  enable  them  to 
obtain  one  of  the  two  incentives.  Comparison  of  the  goal  data  with  the  incentive 
levels  indicates  that  there  was  no  tendency  for  the  two  interventions  to  converge. 
The  goals  that  were  set  were  independent  of  the  incentive  levels.  Out  of  the  25 
opportunities  (5  units  x  5  months)  only  3  goals  set  by  the  units  matched  either  of 
the  incentive  levels.  In  addition,  observation  of  the  goal  setting  sessions 
suggested  that  the  goals  were  being  set  independently  of  those  levels  required  for 
the  incentives. 
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Also,  there  was  no  identifiable  pattern  to  the  goals  in  relation  to  the  in¬ 
centive  levels.  In  8  of  the  25  possible  occasions,  the  goals  were  set  above  the 
highest  incentive  level  (that  needed  to  earn  a  full  day  off).  In  10  of  the  25  oc¬ 
casions,  the  goals  were  set  below  the  lowest  incentive  level  (that  needed  for  a 
half  day  off).  For  the  remaining  7  occasions,  the  goals  were  set  at  or  some¬ 
where  between  the  level  needed  for  the  half  day  and  full  day  off. 

Attitude  Data 


Data  were  also  collected  on  work  attitudes.  A  questionnaire  was  admin¬ 
istered  to  incumbents  and  first  line  supervisors  in  each  of  the  units.  It  mea¬ 
sured  job  satisfaction,  turnover  intentions,  morale,  individual  role  clarity,  clarity 
of  objectives,  and  evaluation  clarity.  Job  satisfaction  was  measured  by  seven 
items  adapted  from  the  Minnesota  Satisfaction  Questionnaire  (Weiss,  Dawis, 
England,  &  Loftquist,  1967).  The  actual  items  used  for  this  and  the  remaining 
scales  are  listed  in  the  appendix.  The  items  for  the  morale  scale  were  adapted 
from  Institute  for  Survey  Research  (ISR)  instruments  (Seashore,  Lawler, 
Mirvis,  &  Cammann,  1983).  Items  for  individual  role  clarity  were  adapted  from 
a  Rizzo,  House,  and  Lirtzman  (1970)  instrument  to  assess  how  well  individuals 
believed  they  understood  their  roles  within  the  organization.  Items  for  turnover 
intentions,  clarity  of  objectives,  and  evaluation  clarity  were  developed  for  this 
project.  The  clarity  of  objectives  portion  was  designed  to  determine  how  well 
individuals  believed  they  understood  individual  and  organizational  priorities.  Fi¬ 
nally,  the  evaluation  clarity  portion  was  designed  to  assess  how  well  they  under¬ 
stood  how  a  particular  degree  of  productivity  might  be  evaluated  by  a  supervisor. 

The  questionnaire  was  administered  four  times:  during  baseline,  and  again 
at  the  end  of  each  treatment.  Internal  consistency  reliability  was  assessed  using 
Cronbach’s  alpha  (Nunnally,  1978)  calculated  for  each  scale  based  on  the  data 
from  the  first  administration.  Internal  consistencies  were  good  for  job  satisfac¬ 
tion  (.82),  morale  (.86),  and  for  role  clarity  (.87).  They  were  somewhat  lower 
for  evaluation  clarity  (.76)  and  for  clarity  of  objectives  (.64),  and  quite  low  for 
the  two-item  turnover  intention  scale  (.39). 

The  treatment  results  are  presented  in  Table  5.  The  means  for  the  four 
conditions  were  analyzed  with  a  one-way  ANOVA,  with  the  four  administrations 
serving  as  a  between-groups  factor.  A  repeated  measures  analysis  was  not  used 
since  the  number  of  individuals  for  all  four  administrations  was  very  small. 
The  means  for  each  treatment  are  indicated,  along  with  the  Ns,  the  p-value  for 
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Table  5 


.  Attitude  Data  by  Treatment 


SCALE  Baseline  Feedback  Goals  Incentives  £  MSW 

Job  Satisfaction 


Means 

23.48 

25.33 

24.23 

22.82 

.05 

25.86 

Ns 

60 

70 

56 

45 

Turnover  Intentions 

Means 

6.9 

6.7 

6.7 

7.0 

.92 

6.02 

Ns 

70 

71 

54 

47 

Morale 

Means 

5.42 

6.90 

6.55 

6.47 

.01 

7.67 

Ns 

74 

72 

55 

47 

Role  Clarity 

Means 

9.70 

9.76 

9.58 

9.33 

.73 

4.53 

Ns 

73 

72 

55 

48 

Clarity  of  Objectives 

Means 

8.32 

8.68 

9.25 

8.54 

.08 

4.25 

Ns 

73 

72 

56 

48 

Evaluation  Clarity  1 

Means 

7.13 

6.20 

4.79 

5.11 

.005 

14.05 

Ns 

67 

55 

48 

36 

1.  The  lower  the  mean,  the  greater  the  evaluation  clarity.  For  all  other 
scales,  the  higher  the  mean,  the  more  positive  the  attitude. 
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the  between-administrations  factor,  and  the  mean  square-within  (MSW)  error 
term  for  the  between-administrations  significance  test. 

The  results  indicate  that  the  measures  of  turnover  intentions,  individual 
role  clarity,  and  clarity  of  objectives  showed  no  significant  changes,  although  the 
means  for  clarity  of  objectives  became  more  positive,  and  the  change  was  close 
to  significant  (p  =  .08).  Job  satisfaction,  morale,  and  evaluation  clarity  became 
significantly  more  positive.  In  general,  the  data  indicated  that  job  attitudes  under 
the  treatments  were  as  favorable  or  more  favorable  than  before. 

Effects  After  The  Departure  Of  The  Research  Team 

Once  the  five-month  incentive  treatment  was  over,  our  on-base  responsi¬ 
bilities  officially  ended.  Although  we  were  on  base  for  a  variety  of  purposes 
after  this  time,  the  units  had  no  commitments  to  continue  the  system.  To 
explore  effects  after  our  departure,  we  shall  address  the  units’  attempts  to 
change  the  system,  our  training  of  unit  personnel  to  take  over  the  system,  the 
units’  ability  to  run  the  system  after  we  left  and,  finally,  the  productivity  scores 
attained  since  we  left. 

A  significant  indication  of  the  value  of  the  system  is  that  each  of  the  five 
units  elected  to  continue  the  system  after  the  researchers  left.  This  meant  that 
they  would  have  to  commit  their  own  resources  to  put  together  the  indicator  data 
and  run  the  programs  producing  the  feedback  reports.  In  addition,  we  were 
asked  by  both  Comm/Nav  and  MS&D  managers  to  develop  the  system  in  other 
units  in  their  respective  squadrons.  Although  we  did  not  have  the  resources  to  do 
so,  it  did  indicate  the  value  that  the  units  placed  on  the  system. 

At  the  end  of  the  formal  incentives  treatment,  units  were  asked  if  they 
wished  to  modify  the  system.  If  they  did  want  to,  we  would  be  there  to  assist 
if  it  became  necessary.  Comm/Nav  and  two  sections  of  MS&D  elected  to  make 
changes.  In  all  three  cases,  the  changes  were  made  to  eliminate  indicators  from 
the  system.  The  indicators  that  were  removed  were  either:  ones  with  very  flat 
contingencies,  indicating  they  were  not  very  important;  activities  that  the  units 
were  no  longer  going  to  perform;  or  indicators  that  were  under  such  good  control 
that  they  no  longer  needed  to  be  measured. 

A  major  strength  of  the  system  is  that  it  can  accommodate  changes  read¬ 
ily.  As  changes  occur  in  policies,  procedures,  or  resources,  changes  will  need  to 
be  made  in  the  system.  This  can  be  done  by  eliminating  indicators,  redefining 
them,  or  altering  the  scaling  of  contingencies.  Thus,  the  system  can  easily  be 
altered  to  changing  conditions.  However,  it  must  be  understood  that  after  such 
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changes  are  made,  the  new  effectiveness  scores  are  no  longer  comparable  to  the 
old  scores.  For  example,  if  indicators  are  dropped,  the  same  actual  productivity 
will  show  up  as  lower  overall  effectiveness  since  some  effectiveness  points  are 
lost  due  to  the  deleted  indicators.  This  makes  the  interpretation  of  effects  over 
time  difficult  until  a  new  "baseline"  is  established.  A  new  baseline  is  estab¬ 
lished  by  taking  the  newly  revised  system  of  indicators  and  contingencies,  and 
calculating  its  overall  ffectiveness  for  several  months  prior  to  the  revision. 
For  example,  if  some  indicators  are  dropped,  it  is  a  straightforward  matter  to 
go  back  to  the  indicator  data  from  prior  months  and  calculate  what  overall  ef¬ 
fectiveness  would  have  been  if  those  indicators  had  not  been  included.  This  then 
becomes  the  new  baseline,  and  the  effectiveness  scores  after  the  change  can  be 
compared  directly  to  this  new  baseline.  If  indicators  are  added,  it  is  a  matter 
of  recalculating  what  the  overall  effectiveness  for  prior  months  would  have  been 
had  these  indicators  been  included.  This  is  a  simple  matter,  provided  historical 
data  are  available. 

One  feature  of  the  system  that  needed  to  be  explored  was  whether  the 
units  would  be  able  to  use  the  system  after  the  researchers  left.  As  part  of 
that  process,  the  units  needed  to  be  able  to  make  changes  in  the  system,  since 
the  need  for  changes  would  undoubtedly  occur  in  the  future.  Thus,  we  were 
particularly  interested  in  their  ability  to  do  this.  In  making  these  changes,  it  be¬ 
came  clear  that  the  management  of  the  units  understood  the  details  of  the  sys¬ 
tem  fairly  well,  and  with  help  from  the  research  team  were  able  to  make  the 
changes.  With  this  help,  they  were  able  to  eliminate  the  indicators  they  wished, 
adjust  the  calculations  of  effectiveness  to  take  the  removal  of  the  indicators  into 
account,  and  recalculate  past  effectiveness  to  establish  a  new  baseline.  Our  as¬ 
sessment  w  3  that  Comm/Nav  could  make  such  changes  completely  on  their  own, 
and  MS&D  could  do  so  with  some  assistance. 

Before  we  left  the  base,  the  units  wanted  us  to  train  them  to  use  the 
system,  so  that  they  could  continue  using  it  after  we  left.  This  proved  to  be  a 
fairly  simple  task  in  Comm/Nav.  By  the  end  of  the  incentives  treatment,  they 
had  already  taken  over  the  collection  of  all  the  data  that  went  into  the  system. 
They  had  only  to  be  trained  to  use  the  microcomputer  programs  designed  to  cal¬ 
culate  effectiveness  scores  and  generate  the  feedback  reports.  This  was  done 
readily,  and  other  than  an  occasional  question,  they  operated  the  system  com¬ 
pletely  on  their  own. 

The  training  in  MS&D  was  more  involved.  Although  their  intent  to  take 
over  the  system  had  been  frequently  expressed  for  some  time,  the  actual  com¬ 
mitment  of  personnel  was  not  made  until  the  end  of  the  incentive  treatment. 
Thus,  the  training  could  not  be  done  gradually  over  many  months,  as  would  have 
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been  optimal.  In  addition,  the  task  of  preparing  the  feedback  reports  in  MS&D 
was  more  difficult  than  in  Comm/Nav.  The  MS&D  reports  required  the  entering 
of  data  showing  the  amount  of  time  it  has  taken  to  move  property  in  the  ware¬ 
house.  Someone  must  take  several  hours  each  month  to  enter  these  data  and  run 
the  program  that  calculates  the  mean  times  for  the  indicator  data.  During  the 
interventions,  this  was  done  by  the  research  team.  After  our  departure,  it  had  to 
be  done  by  MS&D  personnel.  Thus,  they  had  to  learn  how  to  enter  these  data 
and  run  the  program,  and  also  learn  how  to  generate  the  feedback  reports.  Be¬ 
cause  of  these  factors,  it  took  longer  to  train  MS&D  to  take  over  the  system, 
but  as  of  this  writing  they  had  taken  over  the  system  completely. 

A  final  consideration  here  is  what  happened  to  productivity  after  the  units 
took  over  the  system  themselves.  The  results  are  shown  in  Figure  25.  This 
figure  not  only  indicates  the  effects  after  the  units  took  over  system  operation, 
but  also  demonstrates  the  capability  of  the  system  to  generate  a  new  baseline 
when  changes  in  the  system  are  made.  Since  both  Comm/Nav  and  some  sections 
of  MS&D  had  deleted  indicators  from  the  system,  we  had  only  to  recalculate  the 
overall  effectiveness  scores  back  in  time  in  order  to  develop  a  baseline  for 
comparison.  In  this  case,  all  changes  in  the  system  were  made  the  month  fol¬ 
lowing  the  incentive  treatments,  when  our  involvement  in  the  interventions  ended. 
To  develop  a  baseline,  we  recalculated  the  overall  effectiveness  data  for  the  five 
months  of  incentive  treatment.  This  adjusted  score  is  the  overall  effectiveness 
score  that  the  units  would  have  had  during  the  incentive  treatment  if  they  had 
been  under  the  revised  system. 

Based  on  the  revised  system,  the  mean  overall  effectiveness  score  under 
incentives  for  Comm/Nav  was  519,  and  the  mean  after  they  took  over  the  system 
was  556.  For  MS&D,  the  mean  under  incentives  was  1857,  and  the  mean  after 
they  took  over  was  1792.  For  both  units  combined,  the  respective  means  were 
2376  and  2348  during  and  after  incentives,  respectively.  Thus,  the  data  indicate 
that  productivity  remained  approximately  as  high  after  base  personnel  had  respon¬ 
sibility  for  the  system  as  it  had  been  when  the  system  was  operated  by  the  re¬ 
search  team. 


IV.  DISCUSSION 

The  objectives  of  the  present  research  were  presented  earlier  in  this  pa¬ 
per  as  a  series  of  research  questions.  We  will  repeat  these  questions  and  pre¬ 
sent  the  results  pertinent  to  each. 

The  first  research  question  was  whether  the  productivity  measurement 
system  could  be  effectively  developed  in  an  Air  Force  organization.  The  system 
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appears  to  be  a  very  effective  way  of  measuring  productivity  in  the  Air  Force. 
Its  implementation  is  quite  feasible,  unit  personnel  were  cooperative  in  developing 
the  system,  and  it  showed  good  psychometric  properties.  In  addition,  system  de¬ 
velopment  itself  seemed  to  have  a  positive  effect  on  unit  functioning  even  before 
feedback  was  instituted. 

Finally,  the  system  appeared  successful  in  aggregating  productivity  mea¬ 
sures  across  units  so  that  an  integrated  system  could  be  developed  across  the 
four  sections  making  up  a  branch.  This  process  is  actually  quite  simple  once 
the  basic  system  is  developed  in  each  section.  The  application  of  this  aggrega¬ 
tion  to  much  larger  and  more  complex  organizational  units  seems  quite  feasible. 

The  development  of  the  system  and  its  positive  effects  worked  well  on 
units  that  were  quite  different  from  one  another.  The  units  differed  greatly  in 
the  nature  of  the  work,  with  Comm/Nav  doing  repair  of  sophisticated  equipment, 
and  MS&D  operating  a  large  warehouse.  The  technologies  were  quite  different 
between  the  two  units,  as  well  as  between  the  four  sections  of  MS&D.  There 
were  also  great  differences  in  the  type  of  organizational  structure  and  the  work 
flow.  The  units  varied  considerably  in  size  and  the  personnel  varied  consider¬ 
ably  in  academic  as  well  as  in  technical  education.  They  also  differed  in  their 
initial  levels  of  performance.  Yet  with  all  these  differences,  the  system  was 
developed  and  worked  extremely  well  in  each  unit.  This  adds  considerable  sup¬ 
port  to  the  generalizability  of  the  approach. 

The  second  research  question  was  whether  using  the  feedback  resulting 
from  the  productivity  measurement  system  would  increase  productivity.  Feedback 
had  a  very  strong  effect  on  productivity.  An  average  gain  in  productivity  of  50% 
occurred  across  the  five  units  during  feedback.  This  increase  occurred  with 
either  no  change  or  a  decrease  in  manpower.  Competing  explanations  for  the 
positive  results,  such  as  Hawthorne  effects,  increases  in  manpower,  and  in¬ 
creases  brought  about  by  other  organizational  changes,  can  be  ruled  out. 

The  third  research  question  was  whether  goal  setting  and  incentives  would 
increase  productivity  over  feedback.  At  first  inspection,  the  answer  seems  to  be 
that  goal  setting  adds  to  feedback,  but  incentives  add  little  beyond  feedback  plus 
goal  setting.  This  impression  is  based  on  the  overall  changes  in  productivity, 
where  feedback  resulted  in  a  gain  in  productivity  of  50%  over  baseline;  feedback 
plus  goal  setting,  75%;  and  feedback  plus  goal  setting  and  incentives,  76%. 

We  do  not,  however,  believe  that  this  is  the  correct  interpretation.  In¬ 
spection  of  Figure  7  shows  that  across  the  five  units,  there  was  a  large  change 
when  feedback  was  instituted.  The  early  months  of  feedback  showed  a  very 
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strong  improvement  in  productivity.  This  improvement  then  began  to  slow  down, 
and  finally,  the  curve  flattened  out  during  the  last  months.  The  negatively  accel¬ 
erated  curve  from  the  start  of  fee<ft>ack  to  the  end  of  incentives  looks  very 
much  like  a  learning  curve.  This  suggests  that  what  may  have  been  happening 
was  that  the  units  were  continuing  to  learn  how  to  improve  productivity  solely  on 
the  basis  of  the  feedback.  The  possibility  exists  that  the  same  amount  of  in¬ 
crease  over  time  would  have  occurred  if  only  feedback  had  been  used.  One  bit 
of  supporting  evidence  for  this  interpretation  is  that  there  was  no  jump  in  pro¬ 
ductivity  at  the  institution  of  either  goal  setting  or  incentives.  One  would  have 
expected  a  noticeable  increase  in  the  month  these  systems  were  started  if  they 
were  indeed  adding  much  to  productivity. 

A  second  and  very  different  interpretation  has  to  do  with  a  ceiling  effect. 
One  could  argue  that  since  the  units  were  near  their  maximum  possible  levels  of 
overall  effectiveness  by  the  second  month  of  goal  setting,  further  increases  were 
not  really  possible.  Thus,  both  goal  setting  effects  and  incentive  effects  would 
have  been  stronger  if  there  had  been  more  room  for  improvement.  Although  this 
interpretation  is  possible,  a  counter  argument  would  be  that  if  productivity  can  be 
increased  to  its  maximum  with  only  feedback  or  feedback  plus  goal  setting,  in¬ 
centives  are  simply  not  necessary.  It  is  also  possible,  however,  that  goal  setting 
and/or  incentives  would  be  necessary  to  sustain  the  high  levels  of  productivity 
over  time. 

There  is  no  conclusive  way  of  deciding  among  these  different  interpreta¬ 
tions  of  the  data.  The  only  way  to  make  such  conclusions  would  be  to  replicate 
the  study  using  a  much  longer  period  for  each  treatment.  Our  own  best  estimate 
is  that  goal  setting  added  somewhat  to  feedback,  but  not  a  great  deal,  and  that 
incentives  did  not  add  further.  This  conclusion  is  based  on  the  learning  curve 
shape  of  the  productivity  data,  the  lack  of  a  jump  in  productivity  at  the  institution 
of  goal  setting  and  incentives,  and  the  subjective  impressions  of  the  unit  mem¬ 
bers. 

The  fourth  research  question  was  whether  the  system  would  change  work 
attitudes.  The  results  indicate  that  work  attitudes  either  improved  or  stayed  the 
same  across  a  series  of  attitude  dimensions. 

A  fifth  research  question  was  whether  the  system  would  be  seen  as  valu¬ 
able  to  the  users.  Clearly,  it  was.  Attitudes  toward  the  system  ranged  from 
positive  to  very  positive.  The  members  of  the  units  were  quite  proud  of  the 
system  and  their  improvements.  As  (me  indication  of  this,  unit  members  fre¬ 
quently  showed  the  system  to  people  visiting  the  unit.  In  addition,  by  the  end  of 


the  experimental  period,  all  of  the  units  continued  using  the  system  on  their  own, 
and  the  managers  of  all  the  units  wanted  to  expand  it  to  other  units. 

The  final  research  question  was  whether  the  system  would  operate  suc¬ 
cessfully  after  the  departure  of  the  research  team.  The  system  has  operated 
successfully,  at  least  for  the  five  months  for  which  we  have  data  at  this  writ¬ 
ing.  Unit  personnel  learned  to  operate  the  system  successfully,  they  have  contin¬ 
ued  its  use,  and  the  units  have  continued  to  maintain  their  productivity  at  the 
highest  level  that  it  reached  during  the  interventions. 

VI.  CONCLUSIONS 

A  number  of  conclusions  can  be  drawn  from  this  research.  First,  we 
will  discuss  specific  research  conclusions  concerning  the  capabilities  of  the  sys¬ 
tem.  We  will  then  present  our  conclusions  concerning  the  productivity  measure¬ 
ment  system  development  process,  and  conclusions  drawn  about  implementing 
feedback,  goal  setting,  and  incentive  interventions. 

Conclusions  About  Productivity  Measurement  System  Capabilities 

The  productivity  measurement  system  that  was  evaluated  in  the  present  ef¬ 
fort  seems  to  be  a  very  successful  method  of  measuring  productivity  in  Air 
Force  organizations.  Its  use  as  the  basis  for  a  feedback  system  produced  very 
large  increases  in  productivity.  Whether  goal  setting  adds  to  productivity  beyond 
feedback  alone  is  not  clear.  Essentially,  incentives  did  not  increase  productivity 
above  feedback  plus  goal  setting.  The  productivity  interventions  produced  either 
an  improvement  or  no  change  in  various  job  attitudes.  The  units  liked  the  sys¬ 
tem,  and  continued  to  use  it  with  equal  success  after  the  departure  of  the  re¬ 
search  team;  their  high  productivity  has  been  maintained.  Specifically,  the  posi¬ 
tive  productivity  results  continued  for  the  15  months  that  the  research  team  was 
present  on  base,  and  have  continued  for  at  least  5  months  after  that. 

The  development  and  application  of  the  system  also  has  a  number  of  added 
benefits.  It  allows  for  the  direct  comparison  of  the  productivity  of  different 
units  to  each  other.  It  can  be  used  with  both  effectiveness  and  efficiency  ap¬ 
proaches  to  productivity.  In  addition,  it  can  be  applied  to  any  level  of  the  organi¬ 
zation,  allows  for  competition  among  different  units,  helps  identify  the  priorities 
for  increasing  productivity,  and  serves  as  an  excellent  basis  for  evaluating  any 
changes  made  in  the  organization. 
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Conclusions  About  the  Process  of  Developing  Productivity  Systems 


In  doing  this  research,  a  number  of  conclusions  were  drawn  concerning  the 
process  to  be  used  for  developing  productivity  measurement  and  enhancement  pro¬ 
grams  in  Air  Force  enviionments. 

One  is  the  importance  of  having  the  personnel  who  are  going  to  be  using 
the  system  be  heavily  involved  in  its  development.  The  perception  of  unit  per¬ 
sonnel  was  that  some  previous  programs  imposed  from  above  had  not  been  ef¬ 
fective  because  these  programs  were  not  designed  with  an  appreciation  for  their 
unique  needs  and  environment.  It  is  much  more  effective  to  have  heavy  involve¬ 
ment  from  the  personnel  that  are  going  to  use  the  system  so  that  the  final  prod¬ 
uct  will  fit  their  needs  and  they  do  not  feel  that  it  is  yet  one  more  project  im¬ 
posed  from  above. 

It  also  seems  more  effective  to  develop  such  programs  from  the  bottom  of 
the  organization  up.  The  lower  levels  of  supervision  know  the  most  about  the 
functioning  of  the  unit,  the  real  critical  issues,  what  are  reasonable  goals,  and 
what  are  attractive  incentives.  In  addition,  these  are  the  people  that  will  make 
the  system  work.  It  is  important  to  have  their  involvement  and  knowledge  from 
the  start.  It  is  also  important  to  have  higher-level  involvement  to  approve  the 
system.  This  should  be  done  as  the  measurement  system  is  being  developed,  not 
delayed  until  the  system  is  complete.  We  believe  the  technique  of  getting  ap¬ 
proval  on  products  and  indicators  before  starting  contingency  development  proved 
very  valuable.  It  not  only  helped  clarify  policy  earlier  in  the  process,  but  also 
helped  prepare  everyone  for  the  eventual  implementation.  This  approach  gave  all 
levels  of  the  organization  a  chance  to  learn  about  the  system  as  it  was  being  de¬ 
veloped  so  that  they  would  know  how  to  use  it  when  it  was  finished.  It  also 
served  to  generate  considerable  eagerness  at  all  levels  to  receive  the  first  feed¬ 
back  from  the  system. 

We  also  learned  the  importance  of  using  a  multiple  meeting  structure  that 
allows  for  an  iterative  approach  to  the  development  of  the  system.  Our  strategy 
was  to  summarize  the  results  of  each  meeting  and  present  them  at  the  next 
meeting  to  assure  consensus  was  reached.  In  this  way,  unit  personnel  had  time 
to  think  about  the  issues,  discuss  them  among  themselves,  and  be  prepared  to  ap¬ 
proach  the  question  with  fresh  perspectives  at  the  next  meeting.  Personnel  need 
time  to  adjust  to  the  idea  of  completely  capturing  wh*t  they  do  in  a  productivity 
measurement  system.  They  need  to  think  about  how  to  structure  a  feedback  re¬ 
port,  how  many  levels  of  incentives  to  use,  etc.  They  need  to  consider  what  the 
implications  are,  and  be  able  to  discuss  the  issues  among  themselves  thoroughly. 


We  believe  that  a  high  quality  system  such  as  this  could  not  have  been  built  with¬ 
out  multiple  meetings  separated  in  time. 

It  also  proved  very  effective  to  have  unit  personnel  who  developed  the 
measurement  system  defend  it  when  it  was  presented  to  higher  management. 
They  were  much  more  knowledgeable  than  the  researchers  about  the  subtleties  of 
unit  operation,  and  could  better  address  management's  questions  and  concerns. 
Also,  it  gave  them  a  chance  to  present  their  perceptions  of  optimal  policy.  Fi¬ 
nally,  their  sense  of  ownership  of  the  system  was  increased  by  their  defense  of 
it. 

Care  must  be  used  during  development  to  ensure  that  the  resulting  indica¬ 
tors  are  measures  which  are  under  the  control  of  the  unit  (Hurst,  1960).  The 
researchers  frequently  needed  to  remind  unit  personnel  to  assess  whether  they 
had  control  over  a  given  indicator.  Unless  the  unit  has  full  control  over  the 
work  being  performed,  including  measures  of  performance  on  that  work  would 
be  counterproductive  in  that  they  would  decrease  the  motivational  impact  of  the 
system. 

It  is  also  important  that  the  researchers  develop  good  rapport  with  the  op¬ 
erational  personnel.  Some  personnel  were  initially  suspicious  of  our  intentions, 
and  had  questions  about  our  credibility.  Their  experience  had  been  that  outsiders, 
who  were  not  experts  on  their  operations,  had  sometimes  imposed  programs  that 
at  best  resulted  in  extra  work,  and  at  worst  had  a  negative  impact  on  their 
effectiveness.  Taking  the  time  to  really  learn  what  they  did,  and  getting  to 
know  them,  went  a  long  way  toward  decreasing  these  barriers. 

Implementing  Feedback.  Goal  Setting  and  Incentives 

During  implementation  of  the  feedback,  goal  setting,  and  incentive  inter¬ 
ventions,  several  issues  came  up  that  are  issues  for  future  implementations  of 
these  techniques. 

During  feedback,  we  noted  that  management  in  some  cases  seemed  to  fo¬ 
cus  much  more  on  small  amounts  of  negative  productivity  information  rather  than 
on  the  much  larger  amount  of  positive  information.  For  example,  in  some  meet¬ 
ings  where  the  feedback  reports  were  discussed,  a  manager  would  focus  on  the 
few  areas  where  the  unit  decreased  in  effectiveness,  and  largely  ignore  the  ma¬ 
jority  of  areas  where  productivity  had  improved  or  was  quite  high.  This  had  the 
effect  of  making  those  meetings  a  somewhat  punishing  experience  for  unit  mem¬ 
bers,  rather  than  a  positive  experience.  It  is  important  that  management  be 
trained  to  conduct  these  meetings  so  that  they  will  give  recognition  when  it  is 


due,  and  so  that  the  positive  and  negative  information  are  given  appropriate  em¬ 
phasis. 

It  is  extremely  important  that  programs  such  as<  those  used  here  have 
higher  management  support.  This  is  very  important  for  the  continued  operation 
of  the  program.  In  this  regard,  it  is  very  important  that  senior  management  be 
kept  informed  about  the  program  and  fully  understand  it. 

We  did  not  anticipate  the  difficulty  of  keeping  higher  management  support 
of  the  project  when  there  was  turnover  in  these  management  positions.  There 
was  generally  an  initial  resistance  to  the  program  by  the  new  manager.  In  ret¬ 
rospect,  tins  is  not  surprising.  The  program  was  not  developed  by  the  new  man¬ 
ager,  and  he/ she  did  not  have  a  sense  of  ownership  of  it.  Therefore,  it  is  very 
important  to  take  considerable  time  with  the  new  manager  so  that  he/ she  can  be 
convinced  that  the  program  is  indeed  a  sound  one. 

There  were  also  several  issues  associated  with  goals  and  incentives.  We 
have  already  discussed  the  issue  of  "reportable"  and  "non-reportable"  goals,  the 
importance  of  participation  in  the  design  of  the  system,  and  the  importance  of 
joint  supervision  and  incumbent  participation  in  setting  goal  levels.  For  incen¬ 
tives,  we  have  discussed  the  importance  of  the  number  of  incentive  levels  to  be 
used  and  of  ensuring  equity  across  units  in  what  it  takes  to  obtain  the  incentives. 

Another  issue  regarding  incentives  is  that  one  should  expect  resistance  to 
the  use  of  incentives,  especially  from  supervisors.  In  the  present  effort,  we  en¬ 
countered  considerable  resistance  from  supervisors.  The  major  point  made  by 
those  who  were  opposed  to  incentives  was  that  Air  Force  personnel  should  not  be 
rewarded  for  simply  doing  what  they  are  already  supposed  to  do.  There  was 
also  some  indirect  evidence  that  some  supervisors  felt  such  an  incentive  system 
would  undermine  their  power  and  their  own  prerogatives  to  reward  individuals 
and  units  informally.  In  contrast,  the  incumbents  were  much  more  positive  about 
the  incentive  system  than  were  supervisors.  Incumbents  wanted  the  time-off,  but 
more  importantly,  they  wanted  some  tangible  form  of  recognition  for  their  high 
productivity.  There  were,  of  course,  some  supervisors  that  were  in  favor  of 
incentives  and  some  incumbents  who  were  not.  We  would  estimate  that  about 
30%  of  the  supervisors  and  85%  of  the  incumbents  were  in  favor  of  incentives. 

Finally,  care  should  be  taken  with  an  incentive  system  to  ensure  that  man¬ 
agement  does  not  increase  the  level  of  performance  needed  to  obtain  an  incentive 
once  productivity  has  increased.  This  practice  could  easily  lead  to  resentment 
and  a  sense  of  inequity  among  unit  personnel.  In  one  unit,  there  was  an  attempt 
to  do  this  and  the  research  team  managed  to  convince  the  manager  that  this  could 
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have  negative  consequences.  Future  applications  should  be  prepared  for  this 
eventuality. 


Summary 

In  conclusion,  the  present  effort  has  shown  that  productivity  in  units  with 
complex  Air  Force  jobs  can  be  measured  with  this  approach.  The  approach 
proved  very  successful  in  this  application  and  has  many  advantages  that  make  it 
attractive  for  other  applications  as  well.  The  use  of  group-level  feedback  re¬ 
sulting  from  the  productivity  measurement  system  produced  large  increases  in 
productivity,  and  may  well  have  accounted  for  the  majority  of  the  overall  produc¬ 
tivity  gain.  Goal  setting  seemed  to  add  to  productivity,  but  incentives  added  lit¬ 
tle  beyond  feedback  and  goal  setting.  The  interventions  of  feedback  and  goal 
setting  clearly  have  a  place  in  the  techniques  of  productivity  enhancement  in  the 
Air  Force. 
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JOB  SATISFACTION 


Response  Format:  Five-point  Likert  format  with  anchors  of  Very  Dissatisfied, 
Dissatisfied,  Not  Sure,  Satisfied,  and  Very  Satisfied. 


Items: 

1.  The  chance  to  do  something  that  makes  use  of  my  abilities. 

2.  The  way  Air  Force  policies  are  put  into  practice. 

3.  The  freedom  to  use  my  own  judgment. 

4.  The  chance  to  try  my  own  methods  of  doing  the  work. 

5.  The  working  conditions. 

6.  The  praise  I  get  for  doing  a  good  job. 

7.  The  feeling  of  accomplishment  I  get  from  the  job. 


MORALE 


Response  Format:  Five-point  Likert  format  with  anchors  of  Strongly  Disagree, 
Disagree,  Neutral,  Agree,  and  Strongly  Agree. 


Items: 

1.  A  spirit  of  teamwork  exists  between  the  people  in  my  section. 

2.  A  spirit  of  teamwork  exists  between  the  people  in  my  branch. 

3.  The  people  in  the  section  work  together  to  accomplish  the  section’s  objectives. 

4.  The  people  in  the  branch  work  together  to  accomplish  the  branch’s  objectives. 

5.  I  feel  a  sense  of  pride  at  being  a  member  of  this  section. 

6.  I  feel  a  sense  of  pride  at  being  a  member  of  this  branch. 


TURNOVER  INTENTIONS 

Response  Format:  Five-point  Likert  format  with  anchors  of  Strongly  Disagree, 
Disagree,  Neutral,  Agree,  and  Strongly  Agree. 

Items: 

1.  I  plan  to  reenlist  or  extend. 

2.  I  would  like  to  leave  the  Air  Force  next  year. 
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ROLE  CLARITY 


Response  Format:  Five-point  Likert  format  with  anchors  of  Strongly  Disagree, 
Disagree,  Neutral,  Agree,  and  Strongly  Agree. 


Items: 

1.  I  know  what  my  responsibilities  are. 

2.  I  know  exactly  what  is  expected  of  me. 

3.  Explanation  is  clear  of  what  has  to  be  done. 


CLARITY  OF  OBJECTIVES 

Response  Format:  Five-point  Likert  format  with  anchors  of  Strongly  Disagree, 
Disagree,  Neutral,  Agree,  and  Strongly  Agree. 

Items: 

1.  I  understand  which  of  my  work  objectives  are  more  important  than  others. 

2.  The  work  objectives  of  my  section  are  clear  and  specific. 

3.  The  work  objectives  of  the  branch  are  clear  and  specific. 

4.  I  understand  which  of  my  section’s  objectives  are  more  important  than  others. 

5.  I  understand  which  of  the  branch  objectives  are  more  important  than  others. 


?! 
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EVALUATION  CLARITY 


Instructions:  For  each  situation  below,  please  circle  yotjr  estimate  of  the  lowest 
and  highest  ratings  you  think  your  supervisor  might  give  to  you  or  your  work 
group.  For  some  of  these  duties  you  may  be  very  sure  about  your  supervisor’s 
evaluations,  and  the  ’’lowest"  and  the  "highest"  ratings  would  be  very  close.  For 
other  situations,  you  might  be  unsure  what  kind  of  evaluation  your  supervisor 
would  give;  therefore,  there  would  be  a  big  difference  between  the  highest  and 
lowest  possible  evaluations.  Please  rate  what  you  believe  your  supervisor’s  eval¬ 
uation  of  you  and  your  work  group  would  be  for  the  following  monthly  perfor¬ 
mance  data. 

Please  use  the  following  in  deciding  which  number  to  circle.  (The  specific  items 
were  tailored  to  be  relevant  to  each  section.) 

1  =  Worst  possible  2  *  Poor  3  =  Below  average  4  =  Average 
5  =  Above  average  6  =  Good  7  =  Excellent 


1.  On  the  average,  1  2  3  4  5  6  7 

75  percent  of  the 

unfrozen  rejects  1  2  3  4  5  6  7 

were  cleared  in  a  day. 

2.  MS&D  took  an  average  1  2  3  4  5  6  7 

of  2  hours  to  get  all 

priority  2  DOR  items  1  2  3  4  5  6  7 

to  the  customers. 

3.  MS&D  took  an  average  1  2  3  4  5  6  7  highest 

of  2  hours  and  20 

minutes  to  get  all  1  2  3  4  5  6  7  lowest 

priority  3  items  to 
the  customers. 

4.  MS&D  took  an  average  1  2  3  4  5  6  7  highest 

of  48  hours  to  get 

all  priority  4  items  1  2  3  4  5  6  7  lowest 

to  the  customers. 

5.  There  was  a  daily  1  2  3  4  5  6  7  highest 

average  of  100 

delinquent  documents  1  2  3  4  5  6  7  lowest 

charged  to  MS&D  per 

month. 


highest 

lowest 

highest 

lowest 


